2023-02-13 17:54:15,090 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 17:54:15,359 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b 2023-02-13 17:54:15,375 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 17:54:15,376 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c, deleteOnExit=true 2023-02-13 17:54:15,376 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 17:54:15,376 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/test.cache.data in system properties and HBase conf 2023-02-13 17:54:15,377 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 17:54:15,377 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/hadoop.log.dir in system properties and HBase conf 2023-02-13 17:54:15,378 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 17:54:15,379 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 17:54:15,379 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 17:54:15,489 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 17:54:15,862 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 17:54:15,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 17:54:15,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 17:54:15,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 17:54:15,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 17:54:15,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 17:54:15,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 17:54:15,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 17:54:15,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 17:54:15,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 17:54:15,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/nfs.dump.dir in system properties and HBase conf 2023-02-13 17:54:15,868 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/java.io.tmpdir in system properties and HBase conf 2023-02-13 17:54:15,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 17:54:15,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 17:54:15,869 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 17:54:16,350 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 17:54:16,354 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 17:54:16,636 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 17:54:16,787 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 17:54:16,801 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 17:54:16,837 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 17:54:16,867 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/java.io.tmpdir/Jetty_localhost_46259_hdfs____.38716q/webapp 2023-02-13 17:54:17,018 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46259 2023-02-13 17:54:17,029 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 17:54:17,029 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 17:54:17,410 WARN [Listener at localhost/37271] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 17:54:17,522 WARN [Listener at localhost/37271] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 17:54:17,540 WARN [Listener at localhost/37271] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 17:54:17,547 INFO [Listener at localhost/37271] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 17:54:17,551 INFO [Listener at localhost/37271] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/java.io.tmpdir/Jetty_localhost_40907_datanode____.9j347p/webapp 2023-02-13 17:54:17,652 INFO [Listener at localhost/37271] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40907 2023-02-13 17:54:17,958 WARN [Listener at localhost/41067] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 17:54:17,969 WARN [Listener at localhost/41067] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 17:54:17,972 WARN [Listener at localhost/41067] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 17:54:17,975 INFO [Listener at localhost/41067] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 17:54:17,984 INFO [Listener at localhost/41067] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/java.io.tmpdir/Jetty_localhost_45973_datanode____.ju07l1/webapp 2023-02-13 17:54:18,083 INFO [Listener at localhost/41067] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45973 2023-02-13 17:54:18,093 WARN [Listener at localhost/45559] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 17:54:18,108 WARN [Listener at localhost/45559] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 17:54:18,111 WARN [Listener at localhost/45559] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 17:54:18,113 INFO [Listener at localhost/45559] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 17:54:18,120 INFO [Listener at localhost/45559] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/java.io.tmpdir/Jetty_localhost_37103_datanode____d27dvb/webapp 2023-02-13 17:54:18,230 INFO [Listener at localhost/45559] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37103 2023-02-13 17:54:18,240 WARN [Listener at localhost/44157] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 17:54:18,454 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf5ba5c55de547e11: Processing first storage report for DS-8830f639-ff07-45aa-b05f-6d503430b2ff from datanode 14616398-3dff-4b20-9b76-b1842232aebe 2023-02-13 17:54:18,455 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf5ba5c55de547e11: from storage DS-8830f639-ff07-45aa-b05f-6d503430b2ff node DatanodeRegistration(127.0.0.1:45395, datanodeUuid=14616398-3dff-4b20-9b76-b1842232aebe, infoPort=42331, infoSecurePort=0, ipcPort=45559, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,455 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56147009d233d8c1: Processing first storage report for DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d from datanode 9eb8ad26-3d72-4195-a2ef-90d725f84f89 2023-02-13 17:54:18,455 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56147009d233d8c1: from storage DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d node DatanodeRegistration(127.0.0.1:33113, datanodeUuid=9eb8ad26-3d72-4195-a2ef-90d725f84f89, infoPort=41539, infoSecurePort=0, ipcPort=41067, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,455 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x75fc1b36aea0d76: Processing first storage report for DS-0f4093f4-21b5-488c-8d48-8bd7d446688e from datanode 51672cbd-b28f-41e9-adef-3ec58fde0c7e 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x75fc1b36aea0d76: from storage DS-0f4093f4-21b5-488c-8d48-8bd7d446688e node DatanodeRegistration(127.0.0.1:40425, datanodeUuid=51672cbd-b28f-41e9-adef-3ec58fde0c7e, infoPort=40915, infoSecurePort=0, ipcPort=44157, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf5ba5c55de547e11: Processing first storage report for DS-44cc0a01-80c6-464f-857b-f230b97d56de from datanode 14616398-3dff-4b20-9b76-b1842232aebe 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf5ba5c55de547e11: from storage DS-44cc0a01-80c6-464f-857b-f230b97d56de node DatanodeRegistration(127.0.0.1:45395, datanodeUuid=14616398-3dff-4b20-9b76-b1842232aebe, infoPort=42331, infoSecurePort=0, ipcPort=45559, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x56147009d233d8c1: Processing first storage report for DS-a8e22d49-37b3-4591-ae4c-357833cc68d9 from datanode 9eb8ad26-3d72-4195-a2ef-90d725f84f89 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x56147009d233d8c1: from storage DS-a8e22d49-37b3-4591-ae4c-357833cc68d9 node DatanodeRegistration(127.0.0.1:33113, datanodeUuid=9eb8ad26-3d72-4195-a2ef-90d725f84f89, infoPort=41539, infoSecurePort=0, ipcPort=41067, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x75fc1b36aea0d76: Processing first storage report for DS-44848661-0e3d-4678-86cb-63b069a77f85 from datanode 51672cbd-b28f-41e9-adef-3ec58fde0c7e 2023-02-13 17:54:18,456 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x75fc1b36aea0d76: from storage DS-44848661-0e3d-4678-86cb-63b069a77f85 node DatanodeRegistration(127.0.0.1:40425, datanodeUuid=51672cbd-b28f-41e9-adef-3ec58fde0c7e, infoPort=40915, infoSecurePort=0, ipcPort=44157, storageInfo=lv=-57;cid=testClusterID;nsid=565709414;c=1676310856427), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 17:54:18,629 DEBUG [Listener at localhost/44157] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b 2023-02-13 17:54:18,690 INFO [Listener at localhost/44157] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/zookeeper_0, clientPort=64222, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 17:54:18,703 INFO [Listener at localhost/44157] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=64222 2023-02-13 17:54:18,712 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:18,714 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:19,366 INFO [Listener at localhost/44157] util.FSUtils(479): Created version file at hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e with version=8 2023-02-13 17:54:19,366 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/hbase-staging 2023-02-13 17:54:19,654 INFO [Listener at localhost/44157] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 17:54:20,110 INFO [Listener at localhost/44157] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 17:54:20,142 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,143 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,143 INFO [Listener at localhost/44157] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 17:54:20,144 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,144 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 17:54:20,282 INFO [Listener at localhost/44157] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 17:54:20,350 DEBUG [Listener at localhost/44157] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 17:54:20,452 INFO [Listener at localhost/44157] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33435 2023-02-13 17:54:20,462 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,465 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,486 INFO [Listener at localhost/44157] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33435 connecting to ZooKeeper ensemble=127.0.0.1:64222 2023-02-13 17:54:20,526 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:334350x0, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 17:54:20,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:33435-0x1004ed1b5780000 connected 2023-02-13 17:54:20,563 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 17:54:20,564 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:54:20,568 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 17:54:20,575 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33435 2023-02-13 17:54:20,575 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33435 2023-02-13 17:54:20,576 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33435 2023-02-13 17:54:20,576 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33435 2023-02-13 17:54:20,576 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33435 2023-02-13 17:54:20,582 INFO [Listener at localhost/44157] master.HMaster(439): hbase.rootdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e, hbase.cluster.distributed=false 2023-02-13 17:54:20,648 INFO [Listener at localhost/44157] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 17:54:20,649 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,649 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,649 INFO [Listener at localhost/44157] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 17:54:20,649 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,649 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 17:54:20,654 INFO [Listener at localhost/44157] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 17:54:20,657 INFO [Listener at localhost/44157] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34255 2023-02-13 17:54:20,659 INFO [Listener at localhost/44157] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 17:54:20,666 DEBUG [Listener at localhost/44157] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 17:54:20,667 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,669 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,671 INFO [Listener at localhost/44157] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34255 connecting to ZooKeeper ensemble=127.0.0.1:64222 2023-02-13 17:54:20,676 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:342550x0, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 17:54:20,677 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:342550x0, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 17:54:20,681 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:342550x0, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:54:20,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34255-0x1004ed1b5780001 connected 2023-02-13 17:54:20,682 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 17:54:20,684 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34255 2023-02-13 17:54:20,684 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34255 2023-02-13 17:54:20,685 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34255 2023-02-13 17:54:20,685 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34255 2023-02-13 17:54:20,686 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34255 2023-02-13 17:54:20,700 INFO [Listener at localhost/44157] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 17:54:20,700 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,701 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,701 INFO [Listener at localhost/44157] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 17:54:20,701 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,702 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 17:54:20,702 INFO [Listener at localhost/44157] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 17:54:20,704 INFO [Listener at localhost/44157] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46577 2023-02-13 17:54:20,705 INFO [Listener at localhost/44157] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 17:54:20,707 DEBUG [Listener at localhost/44157] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 17:54:20,708 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,714 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,717 INFO [Listener at localhost/44157] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46577 connecting to ZooKeeper ensemble=127.0.0.1:64222 2023-02-13 17:54:20,724 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:465770x0, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 17:54:20,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:46577-0x1004ed1b5780002 connected 2023-02-13 17:54:20,725 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 17:54:20,726 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:54:20,727 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 17:54:20,728 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46577 2023-02-13 17:54:20,729 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46577 2023-02-13 17:54:20,729 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46577 2023-02-13 17:54:20,733 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46577 2023-02-13 17:54:20,733 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46577 2023-02-13 17:54:20,746 INFO [Listener at localhost/44157] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 17:54:20,747 INFO [Listener at localhost/44157] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 17:54:20,749 INFO [Listener at localhost/44157] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39639 2023-02-13 17:54:20,749 INFO [Listener at localhost/44157] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 17:54:20,751 DEBUG [Listener at localhost/44157] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 17:54:20,752 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,754 INFO [Listener at localhost/44157] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,755 INFO [Listener at localhost/44157] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39639 connecting to ZooKeeper ensemble=127.0.0.1:64222 2023-02-13 17:54:20,759 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:396390x0, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 17:54:20,760 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:396390x0, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 17:54:20,761 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:396390x0, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:54:20,762 DEBUG [Listener at localhost/44157] zookeeper.ZKUtil(164): regionserver:396390x0, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 17:54:20,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39639-0x1004ed1b5780003 connected 2023-02-13 17:54:20,766 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39639 2023-02-13 17:54:20,768 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39639 2023-02-13 17:54:20,768 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39639 2023-02-13 17:54:20,769 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39639 2023-02-13 17:54:20,769 DEBUG [Listener at localhost/44157] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39639 2023-02-13 17:54:20,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:20,781 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 17:54:20,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:20,801 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 17:54:20,801 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 17:54:20,801 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:20,801 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 17:54:20,801 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 17:54:20,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 17:54:20,805 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33435,1676310859495 from backup master directory 2023-02-13 17:54:20,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 17:54:20,808 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:20,808 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 17:54:20,809 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 17:54:20,809 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:20,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 17:54:20,813 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 17:54:20,909 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/hbase.id with ID: e1681f74-d1c4-4924-ba48-0e17ee12589e 2023-02-13 17:54:20,953 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 17:54:20,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:21,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x40167ad8 to 127.0.0.1:64222 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 17:54:21,051 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f2927f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 17:54:21,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 17:54:21,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 17:54:21,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 17:54:21,093 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 17:54:21,096 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:54:21,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:54:21,103 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 17:54:21,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store-tmp 2023-02-13 17:54:21,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:21,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 17:54:21,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:54:21,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:54:21,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 17:54:21,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:54:21,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:54:21,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676310861204Disabling compacts and flushes for region at 1676310861204Disabling writes for close at 1676310861205 (+1 ms)Writing region close event to WAL at 1676310861205Closed at 1676310861205 2023-02-13 17:54:21,207 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/WALs/jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:21,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33435%2C1676310859495, suffix=, logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/WALs/jenkins-hbase4.apache.org,33435,1676310859495, archiveDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/oldWALs, maxLogs=10 2023-02-13 17:54:21,289 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK] 2023-02-13 17:54:21,289 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK] 2023-02-13 17:54:21,289 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK] 2023-02-13 17:54:21,298 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:54:21,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/WALs/jenkins-hbase4.apache.org,33435,1676310859495/jenkins-hbase4.apache.org%2C33435%2C1676310859495.1676310861236 2023-02-13 17:54:21,369 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK], DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK], DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK]] 2023-02-13 17:54:21,369 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:21,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:21,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,436 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,443 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 17:54:21,467 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 17:54:21,479 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:21,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,488 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,502 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 17:54:21,506 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:21,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63282335, jitterRate=-0.05701972544193268}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 17:54:21,507 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676310861405Initializing all the Stores at 1676310861407 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310861407Cleaning up temporary data from old regions at 1676310861494 (+87 ms)Cleaning up detritus from prior splits at 1676310861495 (+1 ms)Region opened successfully at 1676310861507 (+12 ms) 2023-02-13 17:54:21,508 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 17:54:21,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:21,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:21,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:21,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 17:54:21,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 35 msec 2023-02-13 17:54:21,568 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 17:54:21,604 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 17:54:21,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 17:54:21,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 17:54:21,645 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 17:54:21,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 17:54:21,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 17:54:21,657 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 17:54:21,659 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:21,661 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 17:54:21,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 17:54:21,674 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 17:54:21,679 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 17:54:21,679 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 17:54:21,679 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 17:54:21,679 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 17:54:21,679 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:21,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,33435,1676310859495, sessionid=0x1004ed1b5780000, setting cluster-up flag (Was=false) 2023-02-13 17:54:21,695 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:21,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 17:54:21,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:21,708 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:21,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 17:54:21,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:21,716 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.hbase-snapshot/.tmp 2023-02-13 17:54:21,773 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(952): ClusterId : e1681f74-d1c4-4924-ba48-0e17ee12589e 2023-02-13 17:54:21,773 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(952): ClusterId : e1681f74-d1c4-4924-ba48-0e17ee12589e 2023-02-13 17:54:21,774 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(952): ClusterId : e1681f74-d1c4-4924-ba48-0e17ee12589e 2023-02-13 17:54:21,779 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 17:54:21,779 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 17:54:21,779 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 17:54:21,786 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 17:54:21,786 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 17:54:21,786 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 17:54:21,786 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 17:54:21,786 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 17:54:21,786 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 17:54:21,792 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 17:54:21,793 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 17:54:21,793 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 17:54:21,794 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ReadOnlyZKClient(139): Connect 0x38179274 to 127.0.0.1:64222 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 17:54:21,794 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ReadOnlyZKClient(139): Connect 0x52db4343 to 127.0.0.1:64222 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 17:54:21,794 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ReadOnlyZKClient(139): Connect 0x55f35cac to 127.0.0.1:64222 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 17:54:21,807 DEBUG [RS:1;jenkins-hbase4:46577] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b34a862, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 17:54:21,808 DEBUG [RS:2;jenkins-hbase4:39639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36d07b06, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 17:54:21,808 DEBUG [RS:1;jenkins-hbase4:46577] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5f0b37dc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 17:54:21,808 DEBUG [RS:2;jenkins-hbase4:39639] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32eff904, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 17:54:21,808 DEBUG [RS:0;jenkins-hbase4:34255] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c1cb768, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 17:54:21,809 DEBUG [RS:0;jenkins-hbase4:34255] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58232833, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 17:54:21,834 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34255 2023-02-13 17:54:21,834 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:39639 2023-02-13 17:54:21,837 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46577 2023-02-13 17:54:21,839 INFO [RS:0;jenkins-hbase4:34255] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 17:54:21,839 INFO [RS:2;jenkins-hbase4:39639] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 17:54:21,840 INFO [RS:0;jenkins-hbase4:34255] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 17:54:21,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:21,839 INFO [RS:1;jenkins-hbase4:46577] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 17:54:21,840 INFO [RS:1;jenkins-hbase4:46577] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 17:54:21,840 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 17:54:21,840 INFO [RS:2;jenkins-hbase4:39639] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 17:54:21,840 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 17:54:21,840 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 17:54:21,843 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33435,1676310859495 with isa=jenkins-hbase4.apache.org/172.31.14.131:34255, startcode=1676310860648 2023-02-13 17:54:21,843 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33435,1676310859495 with isa=jenkins-hbase4.apache.org/172.31.14.131:46577, startcode=1676310860699 2023-02-13 17:54:21,843 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,33435,1676310859495 with isa=jenkins-hbase4.apache.org/172.31.14.131:39639, startcode=1676310860746 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 17:54:21,851 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:21,852 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310891852 2023-02-13 17:54:21,855 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 17:54:21,861 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:21,861 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 17:54:21,866 DEBUG [RS:1;jenkins-hbase4:46577] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 17:54:21,866 DEBUG [RS:0;jenkins-hbase4:34255] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 17:54:21,867 DEBUG [RS:2;jenkins-hbase4:39639] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 17:54:21,867 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 17:54:21,868 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 17:54:21,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 17:54:21,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 17:54:21,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 17:54:21,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 17:54:21,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:21,880 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 17:54:21,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 17:54:21,881 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 17:54:21,884 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 17:54:21,884 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 17:54:21,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676310861886,5,FailOnTimeoutGroup] 2023-02-13 17:54:21,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676310861886,5,FailOnTimeoutGroup] 2023-02-13 17:54:21,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:21,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 17:54:21,889 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:21,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:21,918 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:21,920 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:21,920 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e 2023-02-13 17:54:21,947 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:21,948 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58171, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 17:54:21,948 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55135, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 17:54:21,948 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41369, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 17:54:21,951 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 17:54:21,954 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/info 2023-02-13 17:54:21,955 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 17:54:21,956 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:21,956 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 17:54:21,962 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/rep_barrier 2023-02-13 17:54:21,963 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 17:54:21,963 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33435] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:21,963 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:21,965 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:21,965 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:21,965 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 17:54:21,968 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/table 2023-02-13 17:54:21,968 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 17:54:21,969 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:21,971 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740 2023-02-13 17:54:21,972 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740 2023-02-13 17:54:21,976 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 17:54:21,979 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 17:54:21,985 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:21,986 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73551371, jitterRate=0.09600083529949188}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 17:54:21,986 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676310861948Initializing all the Stores at 1676310861950 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676310861950Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310861950Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676310861950Cleaning up temporary data from old regions at 1676310861974 (+24 ms)Cleaning up detritus from prior splits at 1676310861974Region opened successfully at 1676310861986 (+12 ms) 2023-02-13 17:54:21,986 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 17:54:21,986 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 17:54:21,986 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 17:54:21,986 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 17:54:21,986 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 17:54:21,988 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 17:54:21,989 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e 2023-02-13 17:54:21,989 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676310861986Disabling compacts and flushes for region at 1676310861986Disabling writes for close at 1676310861986Writing region close event to WAL at 1676310861988 (+2 ms)Closed at 1676310861988 2023-02-13 17:54:21,989 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e 2023-02-13 17:54:21,989 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e 2023-02-13 17:54:21,989 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37271 2023-02-13 17:54:21,989 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37271 2023-02-13 17:54:21,989 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 17:54:21,989 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:37271 2023-02-13 17:54:21,990 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 17:54:21,990 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 17:54:21,995 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:21,996 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 17:54:21,999 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:54:22,000 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ZKUtil(162): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,000 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ZKUtil(162): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,000 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ZKUtil(162): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,000 WARN [RS:2;jenkins-hbase4:39639] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 17:54:22,001 WARN [RS:1;jenkins-hbase4:46577] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 17:54:22,001 INFO [RS:2;jenkins-hbase4:39639] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 17:54:22,001 INFO [RS:1;jenkins-hbase4:46577] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 17:54:22,001 WARN [RS:0;jenkins-hbase4:34255] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 17:54:22,002 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,002 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,002 INFO [RS:0;jenkins-hbase4:34255] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 17:54:22,002 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1947): logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,003 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46577,1676310860699] 2023-02-13 17:54:22,003 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34255,1676310860648] 2023-02-13 17:54:22,003 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39639,1676310860746] 2023-02-13 17:54:22,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 17:54:22,029 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ZKUtil(162): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,029 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ZKUtil(162): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,029 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ZKUtil(162): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,030 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ZKUtil(162): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,030 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ZKUtil(162): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,030 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ZKUtil(162): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,030 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ZKUtil(162): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,030 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ZKUtil(162): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,031 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ZKUtil(162): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,038 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 17:54:22,040 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 17:54:22,043 DEBUG [RS:0;jenkins-hbase4:34255] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 17:54:22,043 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 17:54:22,043 DEBUG [RS:2;jenkins-hbase4:39639] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 17:54:22,053 INFO [RS:1;jenkins-hbase4:46577] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 17:54:22,054 INFO [RS:2;jenkins-hbase4:39639] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 17:54:22,053 INFO [RS:0;jenkins-hbase4:34255] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 17:54:22,082 INFO [RS:2;jenkins-hbase4:39639] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 17:54:22,082 INFO [RS:0;jenkins-hbase4:34255] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 17:54:22,082 INFO [RS:1;jenkins-hbase4:46577] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 17:54:22,086 INFO [RS:2;jenkins-hbase4:39639] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 17:54:22,086 INFO [RS:0;jenkins-hbase4:34255] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 17:54:22,087 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,086 INFO [RS:1;jenkins-hbase4:46577] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 17:54:22,087 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,087 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,088 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 17:54:22,088 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 17:54:22,089 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 17:54:22,097 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,097 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,097 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,097 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 17:54:22,098 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 17:54:22,099 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,098 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:1;jenkins-hbase4:46577] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,099 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:2;jenkins-hbase4:39639] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,100 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,101 DEBUG [RS:0;jenkins-hbase4:34255] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 17:54:22,104 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,104 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,104 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,105 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,105 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,105 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,112 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,112 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,112 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,124 INFO [RS:1;jenkins-hbase4:46577] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 17:54:22,127 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46577,1676310860699-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,128 INFO [RS:2;jenkins-hbase4:39639] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 17:54:22,128 INFO [RS:0;jenkins-hbase4:34255] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 17:54:22,129 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39639,1676310860746-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,129 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34255,1676310860648-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,148 INFO [RS:2;jenkins-hbase4:39639] regionserver.Replication(203): jenkins-hbase4.apache.org,39639,1676310860746 started 2023-02-13 17:54:22,148 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39639,1676310860746, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39639, sessionid=0x1004ed1b5780003 2023-02-13 17:54:22,148 INFO [RS:0;jenkins-hbase4:34255] regionserver.Replication(203): jenkins-hbase4.apache.org,34255,1676310860648 started 2023-02-13 17:54:22,148 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 17:54:22,148 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34255,1676310860648, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34255, sessionid=0x1004ed1b5780001 2023-02-13 17:54:22,149 DEBUG [RS:2;jenkins-hbase4:39639] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,149 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 17:54:22,149 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39639,1676310860746' 2023-02-13 17:54:22,149 INFO [RS:1;jenkins-hbase4:46577] regionserver.Replication(203): jenkins-hbase4.apache.org,46577,1676310860699 started 2023-02-13 17:54:22,149 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:22,150 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,46577,1676310860699, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46577, sessionid=0x1004ed1b5780002 2023-02-13 17:54:22,149 DEBUG [RS:0;jenkins-hbase4:34255] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,150 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 17:54:22,151 DEBUG [RS:1;jenkins-hbase4:46577] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,150 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34255,1676310860648' 2023-02-13 17:54:22,151 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:22,151 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46577,1676310860699' 2023-02-13 17:54:22,151 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:22,151 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:22,151 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:22,152 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:22,152 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 17:54:22,152 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 17:54:22,152 DEBUG [RS:2;jenkins-hbase4:39639] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:22,152 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39639,1676310860746' 2023-02-13 17:54:22,152 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 17:54:22,152 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 17:54:22,152 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 17:54:22,152 DEBUG [RS:0;jenkins-hbase4:34255] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:22,153 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 17:54:22,153 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 17:54:22,153 DEBUG [RS:1;jenkins-hbase4:46577] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,153 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34255,1676310860648' 2023-02-13 17:54:22,153 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 17:54:22,153 DEBUG [RS:2;jenkins-hbase4:39639] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 17:54:22,153 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46577,1676310860699' 2023-02-13 17:54:22,154 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 17:54:22,154 DEBUG [RS:0;jenkins-hbase4:34255] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 17:54:22,154 DEBUG [RS:2;jenkins-hbase4:39639] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 17:54:22,154 INFO [RS:2;jenkins-hbase4:39639] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 17:54:22,154 INFO [RS:2;jenkins-hbase4:39639] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 17:54:22,154 DEBUG [RS:0;jenkins-hbase4:34255] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 17:54:22,154 DEBUG [RS:1;jenkins-hbase4:46577] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 17:54:22,154 INFO [RS:0;jenkins-hbase4:34255] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 17:54:22,156 INFO [RS:0;jenkins-hbase4:34255] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 17:54:22,156 DEBUG [RS:1;jenkins-hbase4:46577] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 17:54:22,156 INFO [RS:1;jenkins-hbase4:46577] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 17:54:22,156 INFO [RS:1;jenkins-hbase4:46577] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 17:54:22,193 DEBUG [jenkins-hbase4:33435] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 17:54:22,196 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:22,202 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:22,202 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:22,203 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:22,203 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:22,205 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46577,1676310860699, state=OPENING 2023-02-13 17:54:22,212 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 17:54:22,214 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:22,214 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 17:54:22,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:22,266 INFO [RS:2;jenkins-hbase4:39639] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39639%2C1676310860746, suffix=, logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,39639,1676310860746, archiveDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs, maxLogs=32 2023-02-13 17:54:22,266 INFO [RS:0;jenkins-hbase4:34255] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34255%2C1676310860648, suffix=, logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,34255,1676310860648, archiveDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs, maxLogs=32 2023-02-13 17:54:22,266 INFO [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46577%2C1676310860699, suffix=, logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,46577,1676310860699, archiveDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs, maxLogs=32 2023-02-13 17:54:22,295 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK] 2023-02-13 17:54:22,296 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK] 2023-02-13 17:54:22,296 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK] 2023-02-13 17:54:22,296 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK] 2023-02-13 17:54:22,297 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK] 2023-02-13 17:54:22,297 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK] 2023-02-13 17:54:22,298 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK] 2023-02-13 17:54:22,300 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK] 2023-02-13 17:54:22,300 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK] 2023-02-13 17:54:22,309 INFO [RS:2;jenkins-hbase4:39639] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,39639,1676310860746/jenkins-hbase4.apache.org%2C39639%2C1676310860746.1676310862271 2023-02-13 17:54:22,310 DEBUG [RS:2;jenkins-hbase4:39639] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK], DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK], DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK]] 2023-02-13 17:54:22,314 INFO [RS:0;jenkins-hbase4:34255] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,34255,1676310860648/jenkins-hbase4.apache.org%2C34255%2C1676310860648.1676310862271 2023-02-13 17:54:22,314 INFO [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,46577,1676310860699/jenkins-hbase4.apache.org%2C46577%2C1676310860699.1676310862271 2023-02-13 17:54:22,314 DEBUG [RS:0;jenkins-hbase4:34255] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK], DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK], DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK]] 2023-02-13 17:54:22,314 DEBUG [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK], DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK], DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK]] 2023-02-13 17:54:22,402 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:22,405 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 17:54:22,408 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38632, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 17:54:22,422 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 17:54:22,422 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 17:54:22,426 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46577%2C1676310860699.meta, suffix=.meta, logDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,46577,1676310860699, archiveDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs, maxLogs=32 2023-02-13 17:54:22,451 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK] 2023-02-13 17:54:22,451 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK] 2023-02-13 17:54:22,451 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK] 2023-02-13 17:54:22,458 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/WALs/jenkins-hbase4.apache.org,46577,1676310860699/jenkins-hbase4.apache.org%2C46577%2C1676310860699.meta.1676310862428.meta 2023-02-13 17:54:22,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45395,DS-8830f639-ff07-45aa-b05f-6d503430b2ff,DISK], DatanodeInfoWithStorage[127.0.0.1:40425,DS-0f4093f4-21b5-488c-8d48-8bd7d446688e,DISK], DatanodeInfoWithStorage[127.0.0.1:33113,DS-a2df18b7-7c4b-4a95-87a9-98849ed8186d,DISK]] 2023-02-13 17:54:22,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:22,464 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:22,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 17:54:22,483 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 17:54:22,486 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 17:54:22,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 17:54:22,492 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:22,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 17:54:22,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 17:54:22,496 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 17:54:22,498 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/info 2023-02-13 17:54:22,498 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/info 2023-02-13 17:54:22,498 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 17:54:22,500 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:22,500 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 17:54:22,502 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/rep_barrier 2023-02-13 17:54:22,502 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/rep_barrier 2023-02-13 17:54:22,503 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 17:54:22,504 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:22,504 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 17:54:22,505 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/table 2023-02-13 17:54:22,506 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/table 2023-02-13 17:54:22,506 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 17:54:22,507 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:22,509 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740 2023-02-13 17:54:22,512 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740 2023-02-13 17:54:22,517 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 17:54:22,520 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 17:54:22,522 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58979642, jitterRate=-0.12113484740257263}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 17:54:22,522 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676310862493Writing region info on filesystem at 1676310862493Initializing all the Stores at 1676310862495 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676310862495Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310862495Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676310862496 (+1 ms)Cleaning up temporary data from old regions at 1676310862514 (+18 ms)Cleaning up detritus from prior splits at 1676310862515 (+1 ms)Running coprocessor post-open hooks at 1676310862522 (+7 ms)Region opened successfully at 1676310862522 2023-02-13 17:54:22,547 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676310862394 2023-02-13 17:54:22,569 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 17:54:22,571 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 17:54:22,571 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46577,1676310860699, state=OPEN 2023-02-13 17:54:22,577 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 17:54:22,577 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 17:54:22,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 17:54:22,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46577,1676310860699 in 360 msec 2023-02-13 17:54:22,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 17:54:22,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 578 msec 2023-02-13 17:54:22,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 836 msec 2023-02-13 17:54:22,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676310860797, completionTime=-1 2023-02-13 17:54:22,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 17:54:22,598 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:22,676 DEBUG [hconnection-0x26616879-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 17:54:22,680 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38646, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 17:54:22,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:22,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310922700 2023-02-13 17:54:22,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310982700 2023-02-13 17:54:22,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 101 msec 2023-02-13 17:54:22,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33435,1676310859495-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33435,1676310859495-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33435,1676310859495-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,725 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33435, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,726 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 17:54:22,736 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 17:54:22,743 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 17:54:22,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 17:54:22,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:22,758 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:22,761 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:22,788 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:22,790 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33 empty. 2023-02-13 17:54:22,791 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:22,791 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 17:54:22,833 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:22,835 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4c0f6985b87a7d952cfd36bf98a5ce33, NAME => 'hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:22,856 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:22,856 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 4c0f6985b87a7d952cfd36bf98a5ce33, disabling compactions & flushes 2023-02-13 17:54:22,856 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:22,857 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:22,857 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. after waiting 0 ms 2023-02-13 17:54:22,857 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:22,857 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:22,857 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 4c0f6985b87a7d952cfd36bf98a5ce33: Waiting for close lock at 1676310862856Disabling compacts and flushes for region at 1676310862856Disabling writes for close at 1676310862857 (+1 ms)Writing region close event to WAL at 1676310862857Closed at 1676310862857 2023-02-13 17:54:22,861 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:22,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676310862864"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310862864"}]},"ts":"1676310862864"} 2023-02-13 17:54:22,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:22,910 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:22,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310862911"}]},"ts":"1676310862911"} 2023-02-13 17:54:22,922 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 17:54:22,927 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:22,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:22,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:22,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:22,929 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:22,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4c0f6985b87a7d952cfd36bf98a5ce33, ASSIGN}] 2023-02-13 17:54:22,935 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=4c0f6985b87a7d952cfd36bf98a5ce33, ASSIGN 2023-02-13 17:54:22,936 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=4c0f6985b87a7d952cfd36bf98a5ce33, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:23,088 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:23,089 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:23,090 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676310863089"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310863089"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310863089"}]},"ts":"1676310863089"} 2023-02-13 17:54:23,094 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 4c0f6985b87a7d952cfd36bf98a5ce33, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:23,258 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:23,258 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4c0f6985b87a7d952cfd36bf98a5ce33, NAME => 'hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:23,260 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:23,260 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,260 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:23,260 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,260 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,262 INFO [StoreOpener-4c0f6985b87a7d952cfd36bf98a5ce33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,265 DEBUG [StoreOpener-4c0f6985b87a7d952cfd36bf98a5ce33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/info 2023-02-13 17:54:23,265 DEBUG [StoreOpener-4c0f6985b87a7d952cfd36bf98a5ce33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/info 2023-02-13 17:54:23,265 INFO [StoreOpener-4c0f6985b87a7d952cfd36bf98a5ce33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4c0f6985b87a7d952cfd36bf98a5ce33 columnFamilyName info 2023-02-13 17:54:23,266 INFO [StoreOpener-4c0f6985b87a7d952cfd36bf98a5ce33-1] regionserver.HStore(310): Store=4c0f6985b87a7d952cfd36bf98a5ce33/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:23,268 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,269 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,273 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:54:23,276 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:23,277 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4c0f6985b87a7d952cfd36bf98a5ce33; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63884550, jitterRate=-0.04804602265357971}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 17:54:23,277 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4c0f6985b87a7d952cfd36bf98a5ce33: Running coprocessor pre-open hook at 1676310863261Writing region info on filesystem at 1676310863261Initializing all the Stores at 1676310863262 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676310863262Cleaning up temporary data from old regions at 1676310863270 (+8 ms)Cleaning up detritus from prior splits at 1676310863270Running coprocessor post-open hooks at 1676310863277 (+7 ms)Region opened successfully at 1676310863277 2023-02-13 17:54:23,279 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33., pid=6, masterSystemTime=1676310863249 2023-02-13 17:54:23,282 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:23,282 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:54:23,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:23,285 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676310863283"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310863283"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310863283"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310863283"}]},"ts":"1676310863283"} 2023-02-13 17:54:23,291 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 17:54:23,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 4c0f6985b87a7d952cfd36bf98a5ce33, server=jenkins-hbase4.apache.org,46577,1676310860699 in 194 msec 2023-02-13 17:54:23,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 17:54:23,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=4c0f6985b87a7d952cfd36bf98a5ce33, ASSIGN in 361 msec 2023-02-13 17:54:23,297 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:23,297 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310863297"}]},"ts":"1676310863297"} 2023-02-13 17:54:23,299 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 17:54:23,302 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:23,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 557 msec 2023-02-13 17:54:23,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 17:54:23,362 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 17:54:23,362 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:23,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:23,415 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 17:54:23,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-02-13 17:54:23,431 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:23,442 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 17:54:23,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-13 17:54:23,460 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 17:54:23,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 17:54:23,463 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.654sec 2023-02-13 17:54:23,466 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 17:54:23,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 17:54:23,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 17:54:23,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33435,1676310859495-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 17:54:23,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33435,1676310859495-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 17:54:23,479 DEBUG [Listener at localhost/44157] zookeeper.ReadOnlyZKClient(139): Connect 0x0c9a727f to 127.0.0.1:64222 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 17:54:23,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 17:54:23,516 DEBUG [Listener at localhost/44157] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@75410c3c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 17:54:23,532 DEBUG [hconnection-0x883aab4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 17:54:23,543 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40778, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 17:54:23,552 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:54:23,580 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=168, AvailableMemoryMB=1383 2023-02-13 17:54:23,585 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:23,593 DEBUG [Listener at localhost/44157] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 17:54:23,597 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44722, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 17:54:23,610 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:23,610 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:54:23,611 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:23,617 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:23,620 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:23,621 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:23,624 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:23,625 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab empty. 2023-02-13 17:54:23,627 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:23,627 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 17:54:23,647 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:23,649 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => b84ccdfb3c511e88b7155232983c9cab, NAME => 'testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing b84ccdfb3c511e88b7155232983c9cab, disabling compactions & flushes 2023-02-13 17:54:23,669 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. after waiting 0 ms 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:23,669 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:23,669 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for b84ccdfb3c511e88b7155232983c9cab: Waiting for close lock at 1676310863669Disabling compacts and flushes for region at 1676310863669Disabling writes for close at 1676310863669Writing region close event to WAL at 1676310863669Closed at 1676310863669 2023-02-13 17:54:23,673 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:23,675 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310863675"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310863675"}]},"ts":"1676310863675"} 2023-02-13 17:54:23,678 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:23,679 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:23,680 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310863679"}]},"ts":"1676310863679"} 2023-02-13 17:54:23,683 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 17:54:23,688 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:23,690 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:23,690 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:23,690 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:23,690 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:23,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, ASSIGN}] 2023-02-13 17:54:23,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, ASSIGN 2023-02-13 17:54:23,694 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:23,845 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:23,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b84ccdfb3c511e88b7155232983c9cab, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:23,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310863846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310863846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310863846"}]},"ts":"1676310863846"} 2023-02-13 17:54:23,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure b84ccdfb3c511e88b7155232983c9cab, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:24,007 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,007 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 17:54:24,011 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60428, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 17:54:24,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b84ccdfb3c511e88b7155232983c9cab, NAME => 'testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:24,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:24,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:24,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,020 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,022 DEBUG [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1 2023-02-13 17:54:24,023 DEBUG [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1 2023-02-13 17:54:24,023 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b84ccdfb3c511e88b7155232983c9cab columnFamilyName cf1 2023-02-13 17:54:24,024 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] regionserver.HStore(310): Store=b84ccdfb3c511e88b7155232983c9cab/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:24,024 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,026 DEBUG [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2 2023-02-13 17:54:24,027 DEBUG [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2 2023-02-13 17:54:24,027 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b84ccdfb3c511e88b7155232983c9cab columnFamilyName cf2 2023-02-13 17:54:24,028 INFO [StoreOpener-b84ccdfb3c511e88b7155232983c9cab-1] regionserver.HStore(310): Store=b84ccdfb3c511e88b7155232983c9cab/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:24,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:24,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:24,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:24,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b84ccdfb3c511e88b7155232983c9cab; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74511974, jitterRate=0.11031493544578552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:24,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b84ccdfb3c511e88b7155232983c9cab: Running coprocessor pre-open hook at 1676310864018Writing region info on filesystem at 1676310864018Initializing all the Stores at 1676310864020 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310864020Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310864020Cleaning up temporary data from old regions at 1676310864032 (+12 ms)Cleaning up detritus from prior splits at 1676310864033 (+1 ms)Running coprocessor post-open hooks at 1676310864042 (+9 ms)Region opened successfully at 1676310864042 2023-02-13 17:54:24,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab., pid=11, masterSystemTime=1676310864007 2023-02-13 17:54:24,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=b84ccdfb3c511e88b7155232983c9cab, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310864049"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310864049"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310864049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310864049"}]},"ts":"1676310864049"} 2023-02-13 17:54:24,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 17:54:24,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure b84ccdfb3c511e88b7155232983c9cab, server=jenkins-hbase4.apache.org,39639,1676310860746 in 204 msec 2023-02-13 17:54:24,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 17:54:24,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, ASSIGN in 366 msec 2023-02-13 17:54:24,061 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:24,061 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310864061"}]},"ts":"1676310864061"} 2023-02-13 17:54:24,063 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 17:54:24,068 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:24,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 452 msec 2023-02-13 17:54:24,123 DEBUG [Listener at localhost/44157] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 17:54:24,128 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60440, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 17:54:24,150 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:24,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,158 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,158 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,159 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:24,159 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:24,159 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,159 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,161 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,161 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:24,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,163 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,166 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,166 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,165 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,167 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,167 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,166 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,167 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,167 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,168 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:24,168 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:24,168 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,168 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:24,169 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:54:24,169 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,169 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,170 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,170 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,170 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,170 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,241 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/bb7279c7d3c54548b7291be520ffbef9 2023-02-13 17:54:24,304 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 2023-02-13 17:54:24,313 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/bb7279c7d3c54548b7291be520ffbef9 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9 2023-02-13 17:54:24,323 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:24,326 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 2023-02-13 17:54:24,334 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:24,336 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 167ms, sequenceid=6, compaction requested=false 2023-02-13 17:54:24,337 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,339 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:24,339 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,339 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,342 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,343 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,343 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,344 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,345 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:24,348 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:24,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bb96cee[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:24,348 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,353 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,353 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,353 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,354 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,354 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:24,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,354 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:24,355 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:24,356 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,356 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,356 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,358 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,358 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,358 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,359 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,359 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,359 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,359 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:24,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:24,365 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:24,365 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:24,365 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:24,365 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,365 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,371 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,372 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,374 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,374 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,375 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,386 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,387 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,387 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,387 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,387 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,387 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,387 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:24,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,388 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,390 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:24,390 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:24,490 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:24,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:24,520 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:24,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:24,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,523 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,523 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,524 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:24,524 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:24,524 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,525 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,526 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,526 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,526 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,526 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:24,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,528 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,528 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,529 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,530 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,531 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,531 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:24,531 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:24,531 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,531 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:24,532 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:24,533 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,536 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,536 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,536 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,537 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,537 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,551 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 2023-02-13 17:54:24,580 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/a625e10eaa344f91ac438ae13ba2b82a 2023-02-13 17:54:24,589 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 2023-02-13 17:54:24,597 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:24,599 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/a625e10eaa344f91ac438ae13ba2b82a as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a 2023-02-13 17:54:24,608 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:24,609 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 78ms, sequenceid=19, compaction requested=false 2023-02-13 17:54:24,610 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:24,610 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,610 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:24,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:24,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,612 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,612 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,613 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,613 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:24,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26dc83c[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:24,615 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:24,615 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,616 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,616 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,616 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:24,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:24,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,617 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:24,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,621 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,621 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:24,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:24,627 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:24,627 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:24,627 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:24,627 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,627 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,630 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,631 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,631 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,633 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,644 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,645 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,644 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,645 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:24,645 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,645 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,746 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:24,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:24,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:24,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:24,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,774 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:24,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:24,776 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,776 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,779 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,779 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,779 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,780 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,781 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:24,782 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,782 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,781 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,782 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,783 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,782 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:24,783 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:24,783 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:24,784 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:24,784 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,783 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,784 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:24,784 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,784 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:24,786 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,786 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,787 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,788 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,788 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,788 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,809 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/d4b9e5645a3147e98badb41d350433da 2023-02-13 17:54:24,850 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/76df6aa9e1d34e039de4821fe82f0f94 2023-02-13 17:54:24,861 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/d4b9e5645a3147e98badb41d350433da as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da 2023-02-13 17:54:24,869 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:24,871 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/76df6aa9e1d34e039de4821fe82f0f94 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94 2023-02-13 17:54:24,882 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:24,883 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 99ms, sequenceid=32, compaction requested=true 2023-02-13 17:54:24,883 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:24,883 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:24,883 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:24,884 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:24,885 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:24,886 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,886 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,887 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,887 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:24,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@483b0f32[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:24,889 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:24,889 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,892 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:24,892 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:24,892 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:24,893 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:24,893 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:24,895 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,895 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,895 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,895 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,895 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,895 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,896 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,896 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,896 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,896 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:24,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:24,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:24,901 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:24,901 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:24,901 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:24,901 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,901 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,904 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,903 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,904 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:24,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,905 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:24,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,907 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:24,908 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,908 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,918 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:24,918 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:24,917 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:24,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,918 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:24,918 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:24,918 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:24,918 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,918 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:24,919 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:24,919 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:24,919 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:24,919 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,920 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:24,920 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:24,920 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:24,920 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:24,920 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,020 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:25,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:25,043 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:25,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:25,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,046 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,047 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:25,047 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:25,047 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,047 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,049 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,049 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,049 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,049 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,049 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:25,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,051 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,056 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,056 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,056 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,056 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,056 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,056 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,057 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,058 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,058 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,058 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,058 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:25,058 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,058 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:25,058 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:25,058 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:25,059 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,060 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,060 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,060 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,060 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,060 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,081 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/929a675ea33e4767868965cd1ce23399 2023-02-13 17:54:25,111 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/f8248c21f2e34ad58c083917766f0468 2023-02-13 17:54:25,121 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/929a675ea33e4767868965cd1ce23399 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399 2023-02-13 17:54:25,128 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:25,129 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/f8248c21f2e34ad58c083917766f0468 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468 2023-02-13 17:54:25,137 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:25,139 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 81ms, sequenceid=45, compaction requested=true 2023-02-13 17:54:25,139 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:25,139 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,139 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,140 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,150 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,150 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,150 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:25,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5729344d[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:25,152 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:25,153 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,154 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:25,154 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,154 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,156 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,157 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,157 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,157 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,157 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,157 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,157 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,158 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:25,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:25,161 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:25,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:25,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:25,161 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,161 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,165 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,165 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,166 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,168 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,168 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,168 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,171 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,172 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,172 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,173 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:25,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,172 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:25,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,174 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,274 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:25,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:25,297 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:25,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:25,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,300 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,300 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:25,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:25,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,302 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,303 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,303 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,304 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,306 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,306 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,306 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,307 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,308 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,308 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,309 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,310 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:25,310 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:25,310 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,310 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:25,310 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:25,311 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,313 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,313 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,313 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,314 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,314 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,328 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/8946fb1ff18c4525ae8fbb9466ec45de 2023-02-13 17:54:25,352 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/b2cf7c476bd84bffac38bc75ee78ba01 2023-02-13 17:54:25,360 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/8946fb1ff18c4525ae8fbb9466ec45de as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de 2023-02-13 17:54:25,367 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:25,369 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/b2cf7c476bd84bffac38bc75ee78ba01 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01 2023-02-13 17:54:25,376 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:25,378 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 67ms, sequenceid=58, compaction requested=true 2023-02-13 17:54:25,378 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:25,378 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,378 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:25,379 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:25,379 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,379 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,379 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,380 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,380 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,381 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,381 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:25,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4767fc5a[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:25,383 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:25,383 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,384 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,384 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,384 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,385 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,385 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:25,385 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:25,386 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,386 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,386 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,389 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,389 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,390 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:25,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:25,395 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:25,395 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:25,395 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:25,395 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,395 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:25,396 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,396 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,396 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,397 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,397 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,397 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,397 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,400 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,400 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,401 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,406 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:25,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,406 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,406 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:25,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,407 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,408 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,507 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:25,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:25,529 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:25,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:25,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,532 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,532 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,533 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 17:54:25,533 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 17:54:25,533 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,533 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,535 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,535 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,536 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,535 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,536 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,536 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:25,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:25,537 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,537 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,538 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,539 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 17:54:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 17:54:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:25,539 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,540 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 17:54:25,540 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,540 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:25,540 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:25,540 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,540 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. started... 2023-02-13 17:54:25,541 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:25,542 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,543 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,543 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,543 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,543 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,543 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,559 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/30fcf10c7f744d5e9a44dd81459b2447 2023-02-13 17:54:25,581 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/f63cb5f8a8604cc0a38850f6ba77aafe 2023-02-13 17:54:25,588 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/30fcf10c7f744d5e9a44dd81459b2447 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447 2023-02-13 17:54:25,595 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:25,596 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/f63cb5f8a8604cc0a38850f6ba77aafe as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe 2023-02-13 17:54:25,604 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:25,605 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 64ms, sequenceid=71, compaction requested=true 2023-02-13 17:54:25,605 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b84ccdfb3c511e88b7155232983c9cab: 2023-02-13 17:54:25,606 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,606 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 17:54:25,607 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 17:54:25,608 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 17:54:25,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b27e0a2[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:25,611 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:25,611 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,613 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,613 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,613 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,614 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,614 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 17:54:25,614 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,616 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 17:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,617 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,618 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 17:54:25,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:25,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:25,621 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 17:54:25,622 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:25,622 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:25,622 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,622 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,623 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,624 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:25,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,624 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,625 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:25,626 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,627 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,627 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 17:54:25,631 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:25,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:25,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,632 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:25,631 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:25,632 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:25,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,632 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:25,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:25,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,633 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 17:54:25,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:25,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:25,733 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:25,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:25,756 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,763 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:25,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:25,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:25,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:25,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, UNASSIGN}] 2023-02-13 17:54:25,772 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, UNASSIGN 2023-02-13 17:54:25,773 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b84ccdfb3c511e88b7155232983c9cab, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:25,773 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310865773"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310865773"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310865773"}]},"ts":"1676310865773"} 2023-02-13 17:54:25,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure b84ccdfb3c511e88b7155232983c9cab, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:25,935 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:25,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b84ccdfb3c511e88b7155232983c9cab, disabling compactions & flushes 2023-02-13 17:54:25,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. after waiting 0 ms 2023-02-13 17:54:25,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:25,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b84ccdfb3c511e88b7155232983c9cab 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:54:25,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/88871e8a18544ddda4c1759a2f3715b8 2023-02-13 17:54:25,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/06bc0ee656f54d4899256ac6eb7113b1 2023-02-13 17:54:25,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf1/88871e8a18544ddda4c1759a2f3715b8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8 2023-02-13 17:54:26,003 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:26,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.tmp/cf2/06bc0ee656f54d4899256ac6eb7113b1 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1 2023-02-13 17:54:26,014 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:26,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 79ms, sequenceid=82, compaction requested=true 2023-02-13 17:54:26,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:54:26,038 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:26,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. 2023-02-13 17:54:26,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b84ccdfb3c511e88b7155232983c9cab: Waiting for close lock at 1676310865936Running coprocessor pre-close hooks at 1676310865936Disabling compacts and flushes for region at 1676310865936Disabling writes for close at 1676310865936Obtaining lock to block concurrent updates at 1676310865936Preparing flush snapshotting stores in b84ccdfb3c511e88b7155232983c9cab at 1676310865936Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310865936Flushing stores of testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. at 1676310865937 (+1 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf1: creating writer at 1676310865938 (+1 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf1: appending metadata at 1676310865941 (+3 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf1: closing flushed file at 1676310865941Flushing b84ccdfb3c511e88b7155232983c9cab/cf2: creating writer at 1676310865962 (+21 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf2: appending metadata at 1676310865966 (+4 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf2: closing flushed file at 1676310865966Flushing b84ccdfb3c511e88b7155232983c9cab/cf1: reopening flushed file at 1676310865995 (+29 ms)Flushing b84ccdfb3c511e88b7155232983c9cab/cf2: reopening flushed file at 1676310866007 (+12 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b84ccdfb3c511e88b7155232983c9cab in 79ms, sequenceid=82, compaction requested=true at 1676310866015 (+8 ms)Writing region close event to WAL at 1676310866032 (+17 ms)Running coprocessor post-close hooks at 1676310866038 (+6 ms)Closed at 1676310866039 (+1 ms) 2023-02-13 17:54:26,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=b84ccdfb3c511e88b7155232983c9cab, regionState=CLOSED 2023-02-13 17:54:26,043 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310866042"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310866042"}]},"ts":"1676310866042"} 2023-02-13 17:54:26,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 17:54:26,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure b84ccdfb3c511e88b7155232983c9cab, server=jenkins-hbase4.apache.org,39639,1676310860746 in 269 msec 2023-02-13 17:54:26,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 17:54:26,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=b84ccdfb3c511e88b7155232983c9cab, UNASSIGN in 277 msec 2023-02-13 17:54:26,070 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=b84ccdfb3c511e88b7155232983c9cab, threads=14 2023-02-13 17:54:26,076 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,076 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,076 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,076 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,076 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,080 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,081 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,083 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,083 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,084 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,084 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,088 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,089 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,089 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,125 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447, top=true 2023-02-13 17:54:26,127 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1, top=true 2023-02-13 17:54:26,130 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01, top=true 2023-02-13 17:54:26,195 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de, top=true 2023-02-13 17:54:26,199 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe, top=true 2023-02-13 17:54:26,206 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447 for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,206 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01 for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,206 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,206 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,212 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8, top=true 2023-02-13 17:54:26,229 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1 for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,229 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,271 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,271 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,277 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8 for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,277 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,280 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/.splits/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe for child: 0347879560430b134d799b126cf81358, parent: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,280 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,288 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,289 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,307 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,309 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,317 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,323 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,323 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,326 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 for region: b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:26,326 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region b84ccdfb3c511e88b7155232983c9cab Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:26,382 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:26,384 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:26,387 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676310866387"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676310866387"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676310866387"}]},"ts":"1676310866387"} 2023-02-13 17:54:26,387 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310866387"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310866387"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310866387"}]},"ts":"1676310866387"} 2023-02-13 17:54:26,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310866387"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310866387"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310866387"}]},"ts":"1676310866387"} 2023-02-13 17:54:26,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, ASSIGN}] 2023-02-13 17:54:26,440 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, ASSIGN 2023-02-13 17:54:26,441 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:26,442 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, ASSIGN 2023-02-13 17:54:26,443 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:26,591 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:54:26,593 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cd83e9498f4c7fd91cfbe11625a2fb97, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:26,593 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310866593"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310866593"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310866593"}]},"ts":"1676310866593"} 2023-02-13 17:54:26,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure cd83e9498f4c7fd91cfbe11625a2fb97, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:26,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0347879560430b134d799b126cf81358, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:26,600 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310866600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310866600"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310866600"}]},"ts":"1676310866600"} 2023-02-13 17:54:26,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 0347879560430b134d799b126cf81358, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:26,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cd83e9498f4c7fd91cfbe11625a2fb97, NAME => 'testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 17:54:26,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:26,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:26,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,755 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,756 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1 2023-02-13 17:54:26,757 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1 2023-02-13 17:54:26,757 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd83e9498f4c7fd91cfbe11625a2fb97 columnFamilyName cf1 2023-02-13 17:54:26,791 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-bottom 2023-02-13 17:54:26,799 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-bottom 2023-02-13 17:54:26,807 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-bottom 2023-02-13 17:54:26,819 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-bottom 2023-02-13 17:54:26,820 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(310): Store=cd83e9498f4c7fd91cfbe11625a2fb97/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:26,821 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,822 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2 2023-02-13 17:54:26,822 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2 2023-02-13 17:54:26,823 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd83e9498f4c7fd91cfbe11625a2fb97 columnFamilyName cf2 2023-02-13 17:54:26,842 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-bottom 2023-02-13 17:54:26,851 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-bottom 2023-02-13 17:54:26,860 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-bottom 2023-02-13 17:54:26,872 DEBUG [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-bottom 2023-02-13 17:54:26,872 INFO [StoreOpener-cd83e9498f4c7fd91cfbe11625a2fb97-1] regionserver.HStore(310): Store=cd83e9498f4c7fd91cfbe11625a2fb97/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:26,873 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:26,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:26,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cd83e9498f4c7fd91cfbe11625a2fb97; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62708688, jitterRate=-0.0655677318572998}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:26,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cd83e9498f4c7fd91cfbe11625a2fb97: Running coprocessor pre-open hook at 1676310866754Writing region info on filesystem at 1676310866754Initializing all the Stores at 1676310866755 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310866755Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310866755Cleaning up temporary data from old regions at 1676310866876 (+121 ms)Cleaning up detritus from prior splits at 1676310866877 (+1 ms)Running coprocessor post-open hooks at 1676310866881 (+4 ms)Region opened successfully at 1676310866881 2023-02-13 17:54:26,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97., pid=17, masterSystemTime=1676310866748 2023-02-13 17:54:26,886 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:26,887 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:26,887 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 17:54:26,887 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 17:54:26,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:26,891 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,890 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0347879560430b134d799b126cf81358, NAME => 'testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 17:54:26,891 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): cd83e9498f4c7fd91cfbe11625a2fb97/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:26,891 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): cd83e9498f4c7fd91cfbe11625a2fb97/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:26,891 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of cd83e9498f4c7fd91cfbe11625a2fb97/cf2 in testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,891 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cd83e9498f4c7fd91cfbe11625a2fb97/cf1 in testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:26,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:26,891 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=cd83e9498f4c7fd91cfbe11625a2fb97, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:26,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 0347879560430b134d799b126cf81358 2023-02-13 17:54:26,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:26,892 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310866891"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310866891"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310866891"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310866891"}]},"ts":"1676310866891"} 2023-02-13 17:54:26,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0347879560430b134d799b126cf81358 2023-02-13 17:54:26,892 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/.tmp, totalSize=19.9 K 2023-02-13 17:54:26,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0347879560430b134d799b126cf81358 2023-02-13 17:54:26,892 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/.tmp, totalSize=19.9 K 2023-02-13 17:54:26,897 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310864128 2023-02-13 17:54:26,901 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310864128 2023-02-13 17:54:26,910 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310864497 2023-02-13 17:54:26,909 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0347879560430b134d799b126cf81358 2023-02-13 17:54:26,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-13 17:54:26,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure cd83e9498f4c7fd91cfbe11625a2fb97, server=jenkins-hbase4.apache.org,39639,1676310860746 in 302 msec 2023-02-13 17:54:26,912 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310864497 2023-02-13 17:54:26,913 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310864749 2023-02-13 17:54:26,913 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310864749 2023-02-13 17:54:26,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, ASSIGN in 473 msec 2023-02-13 17:54:26,915 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310865023 2023-02-13 17:54:26,915 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310865023 2023-02-13 17:54:26,917 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1 2023-02-13 17:54:26,917 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1 2023-02-13 17:54:26,918 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0347879560430b134d799b126cf81358 columnFamilyName cf1 2023-02-13 17:54:26,958 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): cd83e9498f4c7fd91cfbe11625a2fb97#cf2#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:26,967 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-top 2023-02-13 17:54:26,969 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): cd83e9498f4c7fd91cfbe11625a2fb97#cf1#compaction#15 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:26,985 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-top 2023-02-13 17:54:26,998 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-top 2023-02-13 17:54:27,034 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-top 2023-02-13 17:54:27,045 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/.tmp/cf1/67e163b7db704c0bbb3d2af7d0c744a9 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/67e163b7db704c0bbb3d2af7d0c744a9 2023-02-13 17:54:27,047 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447 2023-02-13 17:54:27,048 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/.tmp/cf2/96ea43c65e98491c8c4143a42ee939ab as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/96ea43c65e98491c8c4143a42ee939ab 2023-02-13 17:54:27,069 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8 2023-02-13 17:54:27,073 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cd83e9498f4c7fd91cfbe11625a2fb97/cf2 of cd83e9498f4c7fd91cfbe11625a2fb97 into 96ea43c65e98491c8c4143a42ee939ab(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:27,073 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in cd83e9498f4c7fd91cfbe11625a2fb97/cf1 of cd83e9498f4c7fd91cfbe11625a2fb97 into 67e163b7db704c0bbb3d2af7d0c744a9(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:27,074 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for cd83e9498f4c7fd91cfbe11625a2fb97: 2023-02-13 17:54:27,074 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cd83e9498f4c7fd91cfbe11625a2fb97: 2023-02-13 17:54:27,074 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97., storeName=cd83e9498f4c7fd91cfbe11625a2fb97/cf2, priority=12, startTime=1676310866886; duration=0sec 2023-02-13 17:54:27,074 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97., storeName=cd83e9498f4c7fd91cfbe11625a2fb97/cf1, priority=12, startTime=1676310866885; duration=0sec 2023-02-13 17:54:27,075 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:27,075 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:27,076 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de 2023-02-13 17:54:27,076 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(310): Store=0347879560430b134d799b126cf81358/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:27,076 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0347879560430b134d799b126cf81358 2023-02-13 17:54:27,077 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2 2023-02-13 17:54:27,077 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2 2023-02-13 17:54:27,078 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0347879560430b134d799b126cf81358 columnFamilyName cf2 2023-02-13 17:54:27,093 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-top 2023-02-13 17:54:27,097 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-top 2023-02-13 17:54:27,102 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-top 2023-02-13 17:54:27,109 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-top 2023-02-13 17:54:27,117 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1 2023-02-13 17:54:27,123 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01 2023-02-13 17:54:27,129 DEBUG [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe 2023-02-13 17:54:27,129 INFO [StoreOpener-0347879560430b134d799b126cf81358-1] regionserver.HStore(310): Store=0347879560430b134d799b126cf81358/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:27,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358 2023-02-13 17:54:27,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358 2023-02-13 17:54:27,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:27,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0347879560430b134d799b126cf81358 2023-02-13 17:54:27,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0347879560430b134d799b126cf81358; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71714524, jitterRate=0.0686296820640564}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:27,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0347879560430b134d799b126cf81358: Running coprocessor pre-open hook at 1676310866893Writing region info on filesystem at 1676310866893Initializing all the Stores at 1676310866896 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310866896Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310866900 (+4 ms)Cleaning up temporary data from old regions at 1676310867134 (+234 ms)Cleaning up detritus from prior splits at 1676310867134Running coprocessor post-open hooks at 1676310867138 (+4 ms)Region opened successfully at 1676310867138 2023-02-13 17:54:27,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358., pid=18, masterSystemTime=1676310866748 2023-02-13 17:54:27,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:27,140 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:27,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:27,142 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:27,142 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,142 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,142 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): 0347879560430b134d799b126cf81358/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:27,143 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): 0347879560430b134d799b126cf81358/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:27,143 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0347879560430b134d799b126cf81358/cf1 in testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,143 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 0347879560430b134d799b126cf81358/cf2 in testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,143 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/.tmp, totalSize=34.9 K 2023-02-13 17:54:27,143 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/.tmp, totalSize=34.9 K 2023-02-13 17:54:27,144 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310864128 2023-02-13 17:54:27,144 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310864128 2023-02-13 17:54:27,145 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310864497 2023-02-13 17:54:27,145 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310864497 2023-02-13 17:54:27,146 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310864749 2023-02-13 17:54:27,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,146 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310864749 2023-02-13 17:54:27,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,147 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310865023 2023-02-13 17:54:27,147 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0347879560430b134d799b126cf81358, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:27,147 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310865023 2023-02-13 17:54:27,147 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310867147"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310867147"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310867147"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310867147"}]},"ts":"1676310867147"} 2023-02-13 17:54:27,148 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310865276 2023-02-13 17:54:27,148 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310865276 2023-02-13 17:54:27,148 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310865510 2023-02-13 17:54:27,149 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310865510 2023-02-13 17:54:27,149 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310865736 2023-02-13 17:54:27,149 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310865736 2023-02-13 17:54:27,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-13 17:54:27,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 0347879560430b134d799b126cf81358, server=jenkins-hbase4.apache.org,39639,1676310860746 in 549 msec 2023-02-13 17:54:27,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-13 17:54:27,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, ASSIGN in 716 msec 2023-02-13 17:54:27,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 in 1.4010 sec 2023-02-13 17:54:27,195 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): 0347879560430b134d799b126cf81358#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:27,195 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0347879560430b134d799b126cf81358#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:27,223 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/.tmp/cf2/f8d2a02284ad409e89f76991693c22df as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8d2a02284ad409e89f76991693c22df 2023-02-13 17:54:27,225 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/.tmp/cf1/69b9b9fac8ac47d7b707bb8968efa7b1 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/69b9b9fac8ac47d7b707bb8968efa7b1 2023-02-13 17:54:27,231 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0347879560430b134d799b126cf81358/cf2 of 0347879560430b134d799b126cf81358 into f8d2a02284ad409e89f76991693c22df(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:27,231 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0347879560430b134d799b126cf81358: 2023-02-13 17:54:27,231 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358., storeName=0347879560430b134d799b126cf81358/cf2, priority=9, startTime=1676310867140; duration=0sec 2023-02-13 17:54:27,231 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:27,234 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0347879560430b134d799b126cf81358/cf1 of 0347879560430b134d799b126cf81358 into 69b9b9fac8ac47d7b707bb8968efa7b1(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:27,234 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0347879560430b134d799b126cf81358: 2023-02-13 17:54:27,234 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358., storeName=0347879560430b134d799b126cf81358/cf1, priority=9, startTime=1676310867139; duration=0sec 2023-02-13 17:54:27,235 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:27,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:60440 deadline: 1676310927264, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab. is not online on jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:27,399 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 17:54:27,414 INFO [Listener at localhost/44157] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 17:54:27,418 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 17:54:27,456 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:27,468 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 17:54:27,474 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-13 17:54:27,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:27,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310867489"}]},"ts":"1676310867489"} 2023-02-13 17:54:27,492 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 17:54:27,495 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 17:54:27,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 17:54:27,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, UNASSIGN}] 2023-02-13 17:54:27,517 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, UNASSIGN 2023-02-13 17:54:27,523 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=0347879560430b134d799b126cf81358, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:27,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310867521"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310867521"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310867521"}]},"ts":"1676310867521"} 2023-02-13 17:54:27,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 0347879560430b134d799b126cf81358, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:27,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, UNASSIGN 2023-02-13 17:54:27,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=cd83e9498f4c7fd91cfbe11625a2fb97, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:27,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310867539"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310867539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310867539"}]},"ts":"1676310867539"} 2023-02-13 17:54:27,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure cd83e9498f4c7fd91cfbe11625a2fb97, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:27,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 17:54:27,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0347879560430b134d799b126cf81358 2023-02-13 17:54:27,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0347879560430b134d799b126cf81358, disabling compactions & flushes 2023-02-13 17:54:27,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. after waiting 0 ms 2023-02-13 17:54:27,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,699 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8] to archive 2023-02-13 17:54:27,701 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:27,707 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,710 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,712 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,714 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,716 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-8946fb1ff18c4525ae8fbb9466ec45de 2023-02-13 17:54:27,718 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-30fcf10c7f744d5e9a44dd81459b2447 2023-02-13 17:54:27,719 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-88871e8a18544ddda4c1759a2f3715b8 2023-02-13 17:54:27,739 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1] to archive 2023-02-13 17:54:27,741 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:27,742 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,744 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,746 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,747 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,749 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-b2cf7c476bd84bffac38bc75ee78ba01 2023-02-13 17:54:27,750 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-f63cb5f8a8604cc0a38850f6ba77aafe 2023-02-13 17:54:27,751 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/testSplitTableRegionUnevenDaughter=b84ccdfb3c511e88b7155232983c9cab-06bc0ee656f54d4899256ac6eb7113b1 2023-02-13 17:54:27,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:54:27,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:27,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358. 2023-02-13 17:54:27,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0347879560430b134d799b126cf81358: Waiting for close lock at 1676310867687Running coprocessor pre-close hooks at 1676310867687Disabling compacts and flushes for region at 1676310867687Disabling writes for close at 1676310867687Writing region close event to WAL at 1676310867753 (+66 ms)Running coprocessor post-close hooks at 1676310867759 (+6 ms)Closed at 1676310867759 2023-02-13 17:54:27,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0347879560430b134d799b126cf81358 2023-02-13 17:54:27,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:27,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cd83e9498f4c7fd91cfbe11625a2fb97, disabling compactions & flushes 2023-02-13 17:54:27,762 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:27,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:27,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. after waiting 0 ms 2023-02-13 17:54:27,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:27,762 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=0347879560430b134d799b126cf81358, regionState=CLOSED 2023-02-13 17:54:27,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310867762"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310867762"}]},"ts":"1676310867762"} 2023-02-13 17:54:27,763 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399-bottom] to archive 2023-02-13 17:54:27,765 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:27,767 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/bb7279c7d3c54548b7291be520ffbef9.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 17:54:27,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 0347879560430b134d799b126cf81358, server=jenkins-hbase4.apache.org,39639,1676310860746 in 232 msec 2023-02-13 17:54:27,769 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/91341f3e8bbb4967b2fe0f9b159f8df7.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=0347879560430b134d799b126cf81358, UNASSIGN in 267 msec 2023-02-13 17:54:27,771 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/d4b9e5645a3147e98badb41d350433da.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,772 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/929a675ea33e4767868965cd1ce23399.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,774 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468-bottom] to archive 2023-02-13 17:54:27,775 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:27,776 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,778 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/a625e10eaa344f91ac438ae13ba2b82a.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,779 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/76df6aa9e1d34e039de4821fe82f0f94.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,781 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/f8248c21f2e34ad58c083917766f0468.b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:27,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:54:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:27,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97. 2023-02-13 17:54:27,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cd83e9498f4c7fd91cfbe11625a2fb97: Waiting for close lock at 1676310867762Running coprocessor pre-close hooks at 1676310867762Disabling compacts and flushes for region at 1676310867762Disabling writes for close at 1676310867762Writing region close event to WAL at 1676310867782 (+20 ms)Running coprocessor post-close hooks at 1676310867788 (+6 ms)Closed at 1676310867788 2023-02-13 17:54:27,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:27,791 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=cd83e9498f4c7fd91cfbe11625a2fb97, regionState=CLOSED 2023-02-13 17:54:27,791 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676310867791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310867791"}]},"ts":"1676310867791"} 2023-02-13 17:54:27,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 17:54:27,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure cd83e9498f4c7fd91cfbe11625a2fb97, server=jenkins-hbase4.apache.org,39639,1676310860746 in 252 msec 2023-02-13 17:54:27,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-13 17:54:27,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd83e9498f4c7fd91cfbe11625a2fb97, UNASSIGN in 295 msec 2023-02-13 17:54:27,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310867799"}]},"ts":"1676310867799"} 2023-02-13 17:54:27,800 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 17:54:27,802 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 17:54:27,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 17:54:27,805 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 326 msec 2023-02-13 17:54:28,033 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:54:28,088 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 17:54:28,088 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 17:54:28,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 17:54:28,105 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 17:54:28,110 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,118 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,120 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 17:54:28,128 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:28,133 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:28,133 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358 2023-02-13 17:54:28,138 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/recovered.edits] 2023-02-13 17:54:28,138 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/recovered.edits] 2023-02-13 17:54:28,139 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/recovered.edits] 2023-02-13 17:54:28,147 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/67e163b7db704c0bbb3d2af7d0c744a9 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf1/67e163b7db704c0bbb3d2af7d0c744a9 2023-02-13 17:54:28,149 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/30fcf10c7f744d5e9a44dd81459b2447 2023-02-13 17:54:28,149 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/69b9b9fac8ac47d7b707bb8968efa7b1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf1/69b9b9fac8ac47d7b707bb8968efa7b1 2023-02-13 17:54:28,151 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/88871e8a18544ddda4c1759a2f3715b8 2023-02-13 17:54:28,153 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8d2a02284ad409e89f76991693c22df to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/cf2/f8d2a02284ad409e89f76991693c22df 2023-02-13 17:54:28,153 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/96ea43c65e98491c8c4143a42ee939ab to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/cf2/96ea43c65e98491c8c4143a42ee939ab 2023-02-13 17:54:28,154 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/8946fb1ff18c4525ae8fbb9466ec45de 2023-02-13 17:54:28,155 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/91341f3e8bbb4967b2fe0f9b159f8df7 2023-02-13 17:54:28,157 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/929a675ea33e4767868965cd1ce23399 2023-02-13 17:54:28,158 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358/recovered.edits/90.seqid 2023-02-13 17:54:28,158 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97/recovered.edits/90.seqid 2023-02-13 17:54:28,159 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd83e9498f4c7fd91cfbe11625a2fb97 2023-02-13 17:54:28,159 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/0347879560430b134d799b126cf81358 2023-02-13 17:54:28,160 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/bb7279c7d3c54548b7291be520ffbef9 2023-02-13 17:54:28,161 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf1/d4b9e5645a3147e98badb41d350433da 2023-02-13 17:54:28,165 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/06bc0ee656f54d4899256ac6eb7113b1 2023-02-13 17:54:28,167 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/76df6aa9e1d34e039de4821fe82f0f94 2023-02-13 17:54:28,169 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/a625e10eaa344f91ac438ae13ba2b82a 2023-02-13 17:54:28,171 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/b2cf7c476bd84bffac38bc75ee78ba01 2023-02-13 17:54:28,173 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/e8a2b51cc42f4527b8b9d08e24e6dcb8 2023-02-13 17:54:28,175 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f63cb5f8a8604cc0a38850f6ba77aafe 2023-02-13 17:54:28,177 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/cf2/f8248c21f2e34ad58c083917766f0468 2023-02-13 17:54:28,181 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab/recovered.edits/85.seqid 2023-02-13 17:54:28,182 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionUnevenDaughter/b84ccdfb3c511e88b7155232983c9cab 2023-02-13 17:54:28,182 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 17:54:28,185 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,212 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 17:54:28,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 17:54:28,254 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 17:54:28,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 17:54:28,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310868256"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:28,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310868256"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:28,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310868256"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:28,260 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:54:28,260 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b84ccdfb3c511e88b7155232983c9cab, NAME => 'testSplitTableRegionUnevenDaughter,,1676310863616.b84ccdfb3c511e88b7155232983c9cab.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cd83e9498f4c7fd91cfbe11625a2fb97, NAME => 'testSplitTableRegionUnevenDaughter,,1676310865756.cd83e9498f4c7fd91cfbe11625a2fb97.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 0347879560430b134d799b126cf81358, NAME => 'testSplitTableRegionUnevenDaughter,26,1676310865756.0347879560430b134d799b126cf81358.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 17:54:28,260 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 17:54:28,260 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310868260"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:28,262 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 17:54:28,264 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:28,265 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 153 msec 2023-02-13 17:54:28,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 17:54:28,424 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 17:54:28,441 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 358) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57410 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:48072 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:48056 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:47968 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26616879-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42630 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:39639-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42996 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:48008 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42716 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:48322 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57288 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26616879-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26616879-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:47926 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57670 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57648 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57662 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26616879-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57242 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:48314 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=845 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=283 (was 264) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=1294 (was 1383) 2023-02-13 17:54:28,456 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=845, MaxFileDescriptor=60000, SystemLoadAverage=283, ProcessCount=168, AvailableMemoryMB=1294 2023-02-13 17:54:28,456 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:28,462 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:28,463 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:28,465 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:28,466 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:28,467 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:28,469 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,470 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd empty. 2023-02-13 17:54:28,471 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,471 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 17:54:28,483 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:28,485 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => eb1733af0943ef056fdc5749cdda62fd, NAME => 'testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:28,496 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:28,496 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing eb1733af0943ef056fdc5749cdda62fd, disabling compactions & flushes 2023-02-13 17:54:28,496 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,496 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,496 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. after waiting 0 ms 2023-02-13 17:54:28,497 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,497 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,497 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for eb1733af0943ef056fdc5749cdda62fd: Waiting for close lock at 1676310868496Disabling compacts and flushes for region at 1676310868496Disabling writes for close at 1676310868497 (+1 ms)Writing region close event to WAL at 1676310868497Closed at 1676310868497 2023-02-13 17:54:28,499 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:28,500 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310868500"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310868500"}]},"ts":"1676310868500"} 2023-02-13 17:54:28,502 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:28,503 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:28,503 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310868503"}]},"ts":"1676310868503"} 2023-02-13 17:54:28,505 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 17:54:28,508 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:28,509 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:28,509 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:28,509 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:28,509 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:28,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, ASSIGN}] 2023-02-13 17:54:28,511 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, ASSIGN 2023-02-13 17:54:28,512 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:28,662 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:28,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:28,664 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310868663"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310868663"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310868663"}]},"ts":"1676310868663"} 2023-02-13 17:54:28,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:28,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => eb1733af0943ef056fdc5749cdda62fd, NAME => 'testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:28,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:28,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:28,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,824 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,825 DEBUG [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1 2023-02-13 17:54:28,825 DEBUG [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1 2023-02-13 17:54:28,825 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb1733af0943ef056fdc5749cdda62fd columnFamilyName cf1 2023-02-13 17:54:28,826 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] regionserver.HStore(310): Store=eb1733af0943ef056fdc5749cdda62fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:28,826 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,828 DEBUG [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2 2023-02-13 17:54:28,828 DEBUG [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2 2023-02-13 17:54:28,829 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb1733af0943ef056fdc5749cdda62fd columnFamilyName cf2 2023-02-13 17:54:28,830 INFO [StoreOpener-eb1733af0943ef056fdc5749cdda62fd-1] regionserver.HStore(310): Store=eb1733af0943ef056fdc5749cdda62fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:28,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:28,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:28,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:28,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened eb1733af0943ef056fdc5749cdda62fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61639646, jitterRate=-0.08149769902229309}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:28,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for eb1733af0943ef056fdc5749cdda62fd: Running coprocessor pre-open hook at 1676310868822Writing region info on filesystem at 1676310868822Initializing all the Stores at 1676310868823 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310868823Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310868823Cleaning up temporary data from old regions at 1676310868833 (+10 ms)Cleaning up detritus from prior splits at 1676310868833Running coprocessor post-open hooks at 1676310868839 (+6 ms)Region opened successfully at 1676310868839 2023-02-13 17:54:28,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd., pid=27, masterSystemTime=1676310868818 2023-02-13 17:54:28,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:28,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310868846"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310868846"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310868846"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310868846"}]},"ts":"1676310868846"} 2023-02-13 17:54:28,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 17:54:28,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 in 182 msec 2023-02-13 17:54:28,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 17:54:28,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, ASSIGN in 342 msec 2023-02-13 17:54:28,855 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:28,855 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310868855"}]},"ts":"1676310868855"} 2023-02-13 17:54:28,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 17:54:28,860 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:28,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 397 msec 2023-02-13 17:54:28,976 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:28,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:28,978 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:28,978 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:28,979 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:28,979 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:28,979 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:28,979 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,982 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:28,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:28,982 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:28,982 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:28,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:28,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:28,982 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,983 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:28,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:28,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:28,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:28,988 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:28,989 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:28,988 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:28,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:28,989 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:28,990 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:28,990 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:28,990 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:28,991 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:54:28,992 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:28,993 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:28,993 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:28,993 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:28,993 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:28,993 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,003 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 2023-02-13 17:54:29,020 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/b75ce08924064a9c9ab3ff086ea1e429 2023-02-13 17:54:29,026 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 2023-02-13 17:54:29,031 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:29,032 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/b75ce08924064a9c9ab3ff086ea1e429 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 2023-02-13 17:54:29,038 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:29,039 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 48ms, sequenceid=6, compaction requested=false 2023-02-13 17:54:29,039 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 17:54:29,039 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:29,040 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,040 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:29,040 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:29,040 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,040 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,040 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,041 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,041 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,042 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,042 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:29,044 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3bf5286b[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:29,044 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:29,044 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:29,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,047 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,047 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:29,046 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:29,048 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,048 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,049 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:29,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:29,054 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:29,054 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:29,054 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:29,054 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:29,054 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,057 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,057 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,058 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,059 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,061 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,061 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,061 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:29,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,065 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,065 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,066 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:29,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,067 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,068 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,068 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,167 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:29,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:29,186 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:29,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:29,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:29,188 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:29,188 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,189 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:29,189 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:29,189 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,189 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,190 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,190 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,190 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,190 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,191 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,191 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,193 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,194 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,194 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,194 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,194 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,193 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,194 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,194 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,194 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,195 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:29,196 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,196 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:29,196 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,199 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,199 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,199 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,199 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,199 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,208 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/261aac4ccdb5486c8a2abcff6c27ae33 2023-02-13 17:54:29,222 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/95e539f89f6f412f8c9d5c3b48d80c24 2023-02-13 17:54:29,228 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/261aac4ccdb5486c8a2abcff6c27ae33 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 2023-02-13 17:54:29,233 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:29,234 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/95e539f89f6f412f8c9d5c3b48d80c24 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 2023-02-13 17:54:29,238 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:29,239 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 43ms, sequenceid=19, compaction requested=false 2023-02-13 17:54:29,239 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:29,239 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,240 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:29,240 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:29,240 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,240 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,240 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,241 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,241 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,242 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,242 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,244 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:29,244 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2560f804[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:29,244 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:29,244 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,245 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:29,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:29,245 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:29,247 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,247 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,247 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,247 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,251 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,251 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,251 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,251 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,251 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,252 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:29,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:29,255 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:29,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:29,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:29,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:29,255 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,260 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,261 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,261 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:29,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,265 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,265 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,266 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:29,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:29,266 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,367 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:29,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:29,393 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:29,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:29,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:29,395 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:29,395 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,396 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:29,396 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:29,397 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,397 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,398 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,398 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,398 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,399 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,399 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,404 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,404 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,404 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,404 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,404 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:29,405 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:29,405 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,405 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:29,405 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:29,409 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,410 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,416 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/a099b9911d6e4a6d83cbab01040dd7ea 2023-02-13 17:54:29,433 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/56912a261d8d45b7ba1c92be93ed3c3e 2023-02-13 17:54:29,439 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/a099b9911d6e4a6d83cbab01040dd7ea as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea 2023-02-13 17:54:29,443 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:29,444 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/56912a261d8d45b7ba1c92be93ed3c3e as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e 2023-02-13 17:54:29,449 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:29,450 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 45ms, sequenceid=32, compaction requested=true 2023-02-13 17:54:29,450 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:29,450 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:29,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:29,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,451 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,452 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,452 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:29,454 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d1d55de[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:29,454 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:29,454 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:29,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,455 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,455 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:29,456 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:29,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,459 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,459 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,459 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,459 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,459 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,460 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,460 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:29,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:29,464 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:29,464 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:29,464 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:29,464 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:29,464 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,466 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,466 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,471 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,472 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,473 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,474 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,474 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,477 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:29,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,477 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,479 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,479 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,478 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:29,479 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,479 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,480 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:29,480 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:29,580 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:29,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:29,600 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:29,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:29,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:29,604 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:29,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:29,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:29,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:29,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,610 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,610 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,612 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,612 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,613 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,613 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,613 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:29,613 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,613 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,613 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:29,613 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,615 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:29,615 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,618 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,619 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,632 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/cf514e1550df43efa7b560505c5d00aa 2023-02-13 17:54:29,649 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 17:54:29,650 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 17:54:29,650 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:29,650 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 17:54:29,651 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 2023-02-13 17:54:29,658 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/cf514e1550df43efa7b560505c5d00aa as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa 2023-02-13 17:54:29,663 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:29,664 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 2023-02-13 17:54:29,669 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:29,670 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 55ms, sequenceid=45, compaction requested=true 2023-02-13 17:54:29,670 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:29,670 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,670 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:29,670 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:29,670 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,670 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,670 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,673 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,673 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,673 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,673 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:29,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49ab9c2f[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:29,676 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:29,676 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,677 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,677 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,677 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:29,678 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,678 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:29,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:29,680 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,680 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,680 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,680 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,682 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,682 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,682 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,682 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,682 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,683 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:29,688 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:29,688 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:29,688 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:29,688 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:29,688 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:29,688 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:29,690 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,690 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,690 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,690 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,690 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,691 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,691 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,691 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,692 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,694 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,695 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,695 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,698 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,698 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,698 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:29,699 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,699 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,700 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:29,700 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:29,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,800 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:29,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:29,819 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:29,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:29,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:29,822 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,823 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,823 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,823 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,823 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,824 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,824 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:29,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:29,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:29,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,826 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:29,827 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:29,827 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,827 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:29,827 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:29,828 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,829 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,829 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,829 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,829 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,829 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,839 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/9aabdbee243043a68f9705662f14c2dd 2023-02-13 17:54:29,856 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/0e5d7e738c004228a3e6640aa6bce8cd 2023-02-13 17:54:29,861 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/9aabdbee243043a68f9705662f14c2dd as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd 2023-02-13 17:54:29,866 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:29,866 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/0e5d7e738c004228a3e6640aa6bce8cd as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd 2023-02-13 17:54:29,871 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:29,873 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 45ms, sequenceid=58, compaction requested=true 2023-02-13 17:54:29,873 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:29,873 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:29,874 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:29,874 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:29,874 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,874 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:29,874 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:29,875 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,875 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,876 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,876 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:29,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:29,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b8f5301[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:29,878 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:29,878 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:29,879 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,880 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,880 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:29,880 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:29,881 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,881 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,881 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,883 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,883 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,883 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,884 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:29,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,884 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,884 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:29,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:29,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:29,887 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:29,887 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:29,887 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:29,887 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:29,887 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,888 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:29,889 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:29,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:29,890 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:29,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:29,892 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:29,892 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,892 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,899 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,899 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:29,899 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,900 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:29,900 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:29,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:29,900 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:29,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:29,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:29,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:29,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,001 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:30,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:30,019 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:30,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:30,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 17:54:30,022 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 17:54:30,022 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:30,022 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 17:54:30,022 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 17:54:30,022 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,022 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,024 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,024 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,024 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,024 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:30,025 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:30,025 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:30,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:30,026 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:30,026 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:30,026 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:30,026 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. started... 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:30,026 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:30,026 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:30,028 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,028 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:30,045 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/f37abd41a14844f4a623af7a738e56ac 2023-02-13 17:54:30,059 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/6bf46134703546ef88de9cb09ab0852f 2023-02-13 17:54:30,063 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/f37abd41a14844f4a623af7a738e56ac as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac 2023-02-13 17:54:30,068 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:30,069 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/6bf46134703546ef88de9cb09ab0852f as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f 2023-02-13 17:54:30,074 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:30,076 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 50ms, sequenceid=71, compaction requested=true 2023-02-13 17:54:30,076 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for eb1733af0943ef056fdc5749cdda62fd: 2023-02-13 17:54:30,076 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:30,076 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:30,076 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:30,076 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,076 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 17:54:30,076 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 17:54:30,078 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,078 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:30,078 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:30,078 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:30,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:30,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 17:54:30,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@790d30ac[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:30,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 17:54:30,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:30,083 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,083 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:30,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:30,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:30,083 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:30,084 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:30,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:30,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:30,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:30,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:30,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 17:54:30,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:30,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:30,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 17:54:30,088 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 17:54:30,088 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:30,088 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:30,088 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 17:54:30,088 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,091 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:30,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:30,093 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,093 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:30,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,096 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:30,096 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 17:54:30,097 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,099 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,100 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 17:54:30,100 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:30,100 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:30,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:30,100 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 17:54:30,101 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 17:54:30,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:30,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:30,201 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:30,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:30,465 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 17:54:30,466 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:30,467 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,469 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:30,469 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:30,469 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:30,717 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,719 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:30,719 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:30,719 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:30,721 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 17:54:30,721 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:30,721 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:30,721 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:30,722 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:30,722 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:30,726 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 17:54:30,727 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:30,728 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:30,728 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:30,729 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:30,729 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,729 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:30,730 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,730 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,731 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:30,731 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,731 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:30,731 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,732 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,732 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,732 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 17:54:30,733 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:30,733 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:30,734 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310900734 2023-02-13 17:54:30,735 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:30,735 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,739 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:30,739 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:30,739 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:30,739 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310930739 2023-02-13 17:54:30,739 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310990739 2023-02-13 17:54:30,739 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:54:30,739 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:30,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:30,742 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:30,742 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:30,989 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:30,990 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:30,990 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:30,991 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:30,992 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 17:54:30,993 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:30,993 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:30,994 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:30,994 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:30,994 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,995 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:30,995 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,995 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:30,996 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:30,996 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,996 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:30,997 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,997 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,997 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:30,997 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:30,998 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:30,998 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:30,999 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310900999 2023-02-13 17:54:31,000 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:31,000 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,003 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,003 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,004 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:31,004 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310931004 2023-02-13 17:54:31,004 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310991004 2023-02-13 17:54:31,004 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 17:54:31,004 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:31,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN}] 2023-02-13 17:54:31,006 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:31,006 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:31,006 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:31,254 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,254 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:31,254 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:31,254 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:31,254 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:31,255 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:31,255 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:31,255 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:31,255 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:31,256 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:31,257 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 17:54:31,258 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:31,258 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:31,258 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:31,259 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:31,259 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,260 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:31,260 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,260 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,261 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:31,262 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,262 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,262 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,262 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,262 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,262 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:54:31,263 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:31,263 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:31,263 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310901263 2023-02-13 17:54:31,265 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:31,265 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,268 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,268 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,268 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:31,268 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310931268 2023-02-13 17:54:31,268 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310991268 2023-02-13 17:54:31,268 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:31,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN}] 2023-02-13 17:54:31,269 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:31,271 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,271 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,271 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310871271"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310871271"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310871271"}]},"ts":"1676310871271"} 2023-02-13 17:54:31,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:31,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:31,273 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:31,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:31,518 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,519 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:31,519 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:31,519 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:31,520 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:31,520 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:31,520 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:31,520 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:31,520 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:31,521 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:31,522 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:31,522 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:31,523 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:31,523 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:31,523 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:31,524 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:31,524 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,524 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:31,525 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,525 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,525 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:31,526 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,526 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,526 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,526 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,526 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,526 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,526 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,526 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,527 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:31,527 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:31,528 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:31,528 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd to restore RIT 2023-02-13 17:54:31,528 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310901528 2023-02-13 17:54:31,529 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:31,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,531 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,532 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,532 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:31,532 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310931532 2023-02-13 17:54:31,532 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310991532 2023-02-13 17:54:31,532 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:31,532 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310871532"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310871532"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310871532"}]},"ts":"1676310871532"} 2023-02-13 17:54:31,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:31,534 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:31,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:31,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing eb1733af0943ef056fdc5749cdda62fd, disabling compactions & flushes 2023-02-13 17:54:31,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:31,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:31,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. after waiting 0 ms 2023-02-13 17:54:31,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:31,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing eb1733af0943ef056fdc5749cdda62fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:54:31,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/3d97b33340814bd29f377b5733118cb3 2023-02-13 17:54:31,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/d32e610bb609432295aae83523abc445 2023-02-13 17:54:31,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf1/3d97b33340814bd29f377b5733118cb3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 2023-02-13 17:54:31,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:31,723 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/.tmp/cf2/d32e610bb609432295aae83523abc445 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 2023-02-13 17:54:31,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:31,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 41ms, sequenceid=82, compaction requested=true 2023-02-13 17:54:31,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:54:31,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:31,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. 2023-02-13 17:54:31,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for eb1733af0943ef056fdc5749cdda62fd: Waiting for close lock at 1676310871687Running coprocessor pre-close hooks at 1676310871687Disabling compacts and flushes for region at 1676310871687Disabling writes for close at 1676310871688 (+1 ms)Obtaining lock to block concurrent updates at 1676310871688Preparing flush snapshotting stores in eb1733af0943ef056fdc5749cdda62fd at 1676310871688Finished memstore snapshotting testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310871688Flushing stores of testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. at 1676310871689 (+1 ms)Flushing eb1733af0943ef056fdc5749cdda62fd/cf1: creating writer at 1676310871689Flushing eb1733af0943ef056fdc5749cdda62fd/cf1: appending metadata at 1676310871691 (+2 ms)Flushing eb1733af0943ef056fdc5749cdda62fd/cf1: closing flushed file at 1676310871691Flushing eb1733af0943ef056fdc5749cdda62fd/cf2: creating writer at 1676310871702 (+11 ms)Flushing eb1733af0943ef056fdc5749cdda62fd/cf2: appending metadata at 1676310871704 (+2 ms)Flushing eb1733af0943ef056fdc5749cdda62fd/cf2: closing flushed file at 1676310871704Flushing eb1733af0943ef056fdc5749cdda62fd/cf1: reopening flushed file at 1676310871718 (+14 ms)Flushing eb1733af0943ef056fdc5749cdda62fd/cf2: reopening flushed file at 1676310871723 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for eb1733af0943ef056fdc5749cdda62fd in 41ms, sequenceid=82, compaction requested=true at 1676310871729 (+6 ms)Writing region close event to WAL at 1676310871746 (+17 ms)Running coprocessor post-close hooks at 1676310871749 (+3 ms)Closed at 1676310871750 (+1 ms) 2023-02-13 17:54:31,752 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:31,753 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED 2023-02-13 17:54:31,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310871753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310871753"}]},"ts":"1676310871753"} 2023-02-13 17:54:31,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:31,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:31,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:31,782 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,782 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:31,782 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:31,782 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:31,783 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:31,783 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:31,783 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:31,783 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:31,783 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:31,784 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:31,785 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:31,785 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 17:54:31,785 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 17:54:31,785 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:31,785 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:31,786 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:31,786 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:31,786 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,786 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:31,787 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,787 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:31,787 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,788 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,788 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,788 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,788 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,788 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:31,789 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:31,789 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:31,790 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:31,790 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd to restore RIT 2023-02-13 17:54:31,790 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310901790 2023-02-13 17:54:31,791 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:31,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,793 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,793 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:31,793 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:31,793 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310931793 2023-02-13 17:54:31,793 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310991793 2023-02-13 17:54:31,793 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:31,794 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:31,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310871794"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310871794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310871794"}]},"ts":"1676310871794"} 2023-02-13 17:54:31,795 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:31,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 17:54:31,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 in 261 msec 2023-02-13 17:54:31,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:31,799 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:31,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:31,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:32,044 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,044 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:32,044 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:32,044 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:32,044 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:32,045 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:32,045 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:32,045 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:32,045 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:32,045 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:32,047 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:32,047 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 17:54:32,047 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:32,048 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:32,049 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:32,049 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:32,049 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:32,049 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,050 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:32,050 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,050 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,051 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:32,051 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,052 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:32,052 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:32,052 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:32,052 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,052 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,052 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,053 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,053 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,053 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 17:54:32,053 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:32,054 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:32,054 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd to restore RIT 2023-02-13 17:54:32,054 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310902054 2023-02-13 17:54:32,055 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:32,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:32,058 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,058 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,058 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:32,058 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310932058 2023-02-13 17:54:32,058 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310992058 2023-02-13 17:54:32,058 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:32,058 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:32,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:32,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 17:54:32,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN in 791 msec 2023-02-13 17:54:32,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:32,066 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:32,066 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:32,308 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:32,309 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:32,309 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:32,310 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:32,312 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:32,312 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:32,312 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:32,312 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:32,313 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:32,313 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:32,313 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:32,314 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,314 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:32,314 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,315 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,315 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:32,316 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,316 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:32,316 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:32,316 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,317 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,317 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,317 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:54:32,317 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:32,318 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:32,318 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310902318 2023-02-13 17:54:32,320 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:32,320 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,322 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,323 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,323 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:32,323 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310932323 2023-02-13 17:54:32,323 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310992323 2023-02-13 17:54:32,323 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:32,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,327 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:32,337 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=eb1733af0943ef056fdc5749cdda62fd, threads=14 2023-02-13 17:54:32,338 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,338 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,338 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,338 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,338 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,339 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,339 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,342 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,459 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,466 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,472 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,472 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,478 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,490 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,490 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,494 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,495 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,495 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,496 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,500 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,501 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,502 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,502 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region eb1733af0943ef056fdc5749cdda62fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:32,538 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:32,538 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:32,539 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:32,573 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,573 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:32,573 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:32,573 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:32,574 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:32,575 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:32,575 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:32,575 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:32,575 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:32,576 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:32,576 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 17:54:32,577 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:32,577 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:32,577 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:32,578 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:32,578 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:32,578 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:32,578 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,579 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:32,579 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,579 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:32,580 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:32,581 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,581 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,581 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,581 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:32,581 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:32,582 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:32,582 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:32,583 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310902583 2023-02-13 17:54:32,584 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:32,584 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:32,586 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,587 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:32,587 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:32,587 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310932587 2023-02-13 17:54:32,587 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310992587 2023-02-13 17:54:32,587 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:32,598 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=eb1733af0943ef056fdc5749cdda62fd, threads=14 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,600 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,603 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,603 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,603 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,604 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,604 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,604 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,607 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,740 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,768 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,769 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,778 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,816 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,817 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,817 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,821 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,821 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,824 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,824 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,828 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:32,828 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:33,138 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 for region: eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:33,139 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region eb1733af0943ef056fdc5749cdda62fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:33,178 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:33,184 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:33,186 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:33,186 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:33,186 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:33,186 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:33,262 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:54:33,337 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,337 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:33,338 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:33,338 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:33,339 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:33,339 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:33,339 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:33,339 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:33,339 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:33,340 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:33,341 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 17:54:33,341 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:33,341 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:33,342 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:33,342 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:33,342 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:33,343 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:33,343 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,344 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:33,344 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,344 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,344 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:33,345 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,345 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,345 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:33,345 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:33,346 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:33,346 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:33,347 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:33,347 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310903347 2023-02-13 17:54:33,348 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:33,348 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,351 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:33,351 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:33,351 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:33,351 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310933351 2023-02-13 17:54:33,351 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310993351 2023-02-13 17:54:33,351 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:33,354 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:33,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:33,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:33,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:33,601 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:33,602 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:33,602 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:33,603 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:33,604 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 17:54:33,604 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:33,604 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:33,604 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:33,605 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:33,605 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:33,605 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:33,605 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,606 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:33,606 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,606 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:33,606 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:33,607 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,607 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN 2023-02-13 17:54:33,607 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:33,607 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:33,607 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,607 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,607 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,607 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=eb1733af0943ef056fdc5749cdda62fd, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:33,608 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure eb1733af0943ef056fdc5749cdda62fd, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:33,608 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:33,608 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:33,609 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:33,609 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310903609 2023-02-13 17:54:33,610 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:33,611 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,612 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:33,613 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=eb1733af0943ef056fdc5749cdda62fd, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:33,613 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:33,613 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310933613 2023-02-13 17:54:33,613 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310993613 2023-02-13 17:54:33,613 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:33,615 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310873615"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676310873615"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676310873615"}]},"ts":"1676310873615"} 2023-02-13 17:54:33,615 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310873615"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310873615"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310873615"}]},"ts":"1676310873615"} 2023-02-13 17:54:33,615 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310873615"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310873615"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310873615"}]},"ts":"1676310873615"} 2023-02-13 17:54:33,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, ASSIGN}] 2023-02-13 17:54:33,622 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, ASSIGN 2023-02-13 17:54:33,623 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:33,624 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, ASSIGN 2023-02-13 17:54:33,624 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:33,773 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:54:33,775 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=357e17d9996b2cb73c13bcd478ff4f03, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:33,775 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310873775"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310873775"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310873775"}]},"ts":"1676310873775"} 2023-02-13 17:54:33,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 357e17d9996b2cb73c13bcd478ff4f03, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:33,780 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=abc074cb651d61f0e94ec5028574f5d0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:33,780 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310873780"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310873780"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310873780"}]},"ts":"1676310873780"} 2023-02-13 17:54:33,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure abc074cb651d61f0e94ec5028574f5d0, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:33,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:33,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => abc074cb651d61f0e94ec5028574f5d0, NAME => 'testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 17:54:33,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:33,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:33,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,934 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,935 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1 2023-02-13 17:54:33,935 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1 2023-02-13 17:54:33,935 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abc074cb651d61f0e94ec5028574f5d0 columnFamilyName cf1 2023-02-13 17:54:33,949 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-top 2023-02-13 17:54:33,953 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-top 2023-02-13 17:54:33,957 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-top 2023-02-13 17:54:33,961 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-top 2023-02-13 17:54:33,965 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-top 2023-02-13 17:54:33,969 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-top 2023-02-13 17:54:33,973 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-top 2023-02-13 17:54:33,973 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(310): Store=abc074cb651d61f0e94ec5028574f5d0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:33,973 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:33,974 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2 2023-02-13 17:54:33,974 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2 2023-02-13 17:54:33,974 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region abc074cb651d61f0e94ec5028574f5d0 columnFamilyName cf2 2023-02-13 17:54:33,988 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-top 2023-02-13 17:54:33,992 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-top 2023-02-13 17:54:33,996 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-top 2023-02-13 17:54:34,000 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-top 2023-02-13 17:54:34,004 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-top 2023-02-13 17:54:34,009 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-top 2023-02-13 17:54:34,013 DEBUG [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-top 2023-02-13 17:54:34,014 INFO [StoreOpener-abc074cb651d61f0e94ec5028574f5d0-1] regionserver.HStore(310): Store=abc074cb651d61f0e94ec5028574f5d0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:34,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:34,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened abc074cb651d61f0e94ec5028574f5d0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60644094, jitterRate=-0.09633257985115051}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:34,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for abc074cb651d61f0e94ec5028574f5d0: Running coprocessor pre-open hook at 1676310873933Writing region info on filesystem at 1676310873933Initializing all the Stores at 1676310873934 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310873934Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310873934Cleaning up temporary data from old regions at 1676310874016 (+82 ms)Cleaning up detritus from prior splits at 1676310874016Running coprocessor post-open hooks at 1676310874019 (+3 ms)Region opened successfully at 1676310874019 2023-02-13 17:54:34,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0., pid=34, masterSystemTime=1676310873929 2023-02-13 17:54:34,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,021 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:34,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:34,022 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,022 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:34,022 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1912): abc074cb651d61f0e94ec5028574f5d0/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:34,022 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,022 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of abc074cb651d61f0e94ec5028574f5d0/cf1 in testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,022 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1912): abc074cb651d61f0e94ec5028574f5d0/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:34,022 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2258): Starting compaction of abc074cb651d61f0e94ec5028574f5d0/cf2 in testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,022 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/.tmp, totalSize=34.9 K 2023-02-13 17:54:34,023 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/.tmp, totalSize=34.9 K 2023-02-13 17:54:34,023 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310868968 2023-02-13 17:54:34,023 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310868968 2023-02-13 17:54:34,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 357e17d9996b2cb73c13bcd478ff4f03, NAME => 'testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 17:54:34,024 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310869169 2023-02-13 17:54:34,024 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310869169 2023-02-13 17:54:34,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:34,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,025 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310869369 2023-02-13 17:54:34,025 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310869369 2023-02-13 17:54:34,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:34,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=abc074cb651d61f0e94ec5028574f5d0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:34,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874025"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310874025"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310874025"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310874025"}]},"ts":"1676310874025"} 2023-02-13 17:54:34,026 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310869583 2023-02-13 17:54:34,026 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310869583 2023-02-13 17:54:34,027 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310869802 2023-02-13 17:54:34,027 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310869802 2023-02-13 17:54:34,027 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310870003 2023-02-13 17:54:34,027 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,028 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310870003 2023-02-13 17:54:34,028 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310870203 2023-02-13 17:54:34,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-13 17:54:34,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure abc074cb651d61f0e94ec5028574f5d0, server=jenkins-hbase4.apache.org,46577,1676310860699 in 245 msec 2023-02-13 17:54:34,030 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310870203 2023-02-13 17:54:34,031 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1 2023-02-13 17:54:34,031 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1 2023-02-13 17:54:34,032 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 357e17d9996b2cb73c13bcd478ff4f03 columnFamilyName cf1 2023-02-13 17:54:34,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, ASSIGN in 411 msec 2023-02-13 17:54:34,046 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] throttle.PressureAwareThroughputController(145): abc074cb651d61f0e94ec5028574f5d0#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:34,049 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] throttle.PressureAwareThroughputController(145): abc074cb651d61f0e94ec5028574f5d0#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:34,051 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-bottom 2023-02-13 17:54:34,055 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-bottom 2023-02-13 17:54:34,059 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-bottom 2023-02-13 17:54:34,063 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-bottom 2023-02-13 17:54:34,073 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-bottom 2023-02-13 17:54:34,083 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-bottom 2023-02-13 17:54:34,087 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/.tmp/cf2/2731c34c9785403e900774b4f7da89de as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/2731c34c9785403e900774b4f7da89de 2023-02-13 17:54:34,087 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/.tmp/cf1/b126c6cc9d2d42eebfe23c9b15642ec4 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/b126c6cc9d2d42eebfe23c9b15642ec4 2023-02-13 17:54:34,094 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-bottom 2023-02-13 17:54:34,095 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(310): Store=357e17d9996b2cb73c13bcd478ff4f03/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:34,095 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,096 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2 2023-02-13 17:54:34,096 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2 2023-02-13 17:54:34,097 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 357e17d9996b2cb73c13bcd478ff4f03 columnFamilyName cf2 2023-02-13 17:54:34,098 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in abc074cb651d61f0e94ec5028574f5d0/cf2 of abc074cb651d61f0e94ec5028574f5d0 into 2731c34c9785403e900774b4f7da89de(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:34,098 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in abc074cb651d61f0e94ec5028574f5d0/cf1 of abc074cb651d61f0e94ec5028574f5d0 into b126c6cc9d2d42eebfe23c9b15642ec4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:34,098 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for abc074cb651d61f0e94ec5028574f5d0: 2023-02-13 17:54:34,098 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for abc074cb651d61f0e94ec5028574f5d0: 2023-02-13 17:54:34,098 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0., storeName=abc074cb651d61f0e94ec5028574f5d0/cf2, priority=9, startTime=1676310874021; duration=0sec 2023-02-13 17:54:34,098 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0., storeName=abc074cb651d61f0e94ec5028574f5d0/cf1, priority=9, startTime=1676310874020; duration=0sec 2023-02-13 17:54:34,098 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,099 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,111 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-bottom 2023-02-13 17:54:34,115 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-bottom 2023-02-13 17:54:34,119 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-bottom 2023-02-13 17:54:34,123 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-bottom 2023-02-13 17:54:34,126 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-bottom 2023-02-13 17:54:34,131 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-bottom 2023-02-13 17:54:34,136 DEBUG [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-bottom 2023-02-13 17:54:34,136 INFO [StoreOpener-357e17d9996b2cb73c13bcd478ff4f03-1] regionserver.HStore(310): Store=357e17d9996b2cb73c13bcd478ff4f03/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:34,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:34,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 357e17d9996b2cb73c13bcd478ff4f03; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71560821, jitterRate=0.06633932888507843}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:34,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 357e17d9996b2cb73c13bcd478ff4f03: Running coprocessor pre-open hook at 1676310874025Writing region info on filesystem at 1676310874025Initializing all the Stores at 1676310874027 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310874027Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310874027Cleaning up temporary data from old regions at 1676310874139 (+112 ms)Cleaning up detritus from prior splits at 1676310874139Running coprocessor post-open hooks at 1676310874141 (+2 ms)Region opened successfully at 1676310874142 (+1 ms) 2023-02-13 17:54:34,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03., pid=33, masterSystemTime=1676310873929 2023-02-13 17:54:34,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:34,143 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:34,143 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,144 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1912): 357e17d9996b2cb73c13bcd478ff4f03/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,144 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1912): 357e17d9996b2cb73c13bcd478ff4f03/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 357e17d9996b2cb73c13bcd478ff4f03/cf1 in testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 357e17d9996b2cb73c13bcd478ff4f03/cf2 in testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/.tmp, totalSize=34.9 K 2023-02-13 17:54:34,144 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/.tmp, totalSize=34.9 K 2023-02-13 17:54:34,145 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310868968 2023-02-13 17:54:34,145 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310868968 2023-02-13 17:54:34,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,145 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310869169 2023-02-13 17:54:34,145 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310869169 2023-02-13 17:54:34,146 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310869369 2023-02-13 17:54:34,146 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=357e17d9996b2cb73c13bcd478ff4f03, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:34,146 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310869369 2023-02-13 17:54:34,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874146"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310874146"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310874146"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310874146"}]},"ts":"1676310874146"} 2023-02-13 17:54:34,146 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310869583 2023-02-13 17:54:34,147 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310869583 2023-02-13 17:54:34,147 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310869802 2023-02-13 17:54:34,147 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310869802 2023-02-13 17:54:34,147 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310870003 2023-02-13 17:54:34,147 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310870003 2023-02-13 17:54:34,148 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310870203 2023-02-13 17:54:34,148 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310870203 2023-02-13 17:54:34,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-13 17:54:34,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 357e17d9996b2cb73c13bcd478ff4f03, server=jenkins-hbase4.apache.org,46577,1676310860699 in 371 msec 2023-02-13 17:54:34,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 17:54:34,154 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, ASSIGN in 533 msec 2023-02-13 17:54:34,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 in 3.6880 sec 2023-02-13 17:54:34,165 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] throttle.PressureAwareThroughputController(145): 357e17d9996b2cb73c13bcd478ff4f03#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:34,165 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] throttle.PressureAwareThroughputController(145): 357e17d9996b2cb73c13bcd478ff4f03#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:34,188 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/.tmp/cf1/124dec46945a484b88a49761aa9087fc as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/124dec46945a484b88a49761aa9087fc 2023-02-13 17:54:34,190 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/.tmp/cf2/b8ac9b395fa744ea9b543c6e81e7a488 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b8ac9b395fa744ea9b543c6e81e7a488 2023-02-13 17:54:34,193 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 357e17d9996b2cb73c13bcd478ff4f03/cf1 of 357e17d9996b2cb73c13bcd478ff4f03 into 124dec46945a484b88a49761aa9087fc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:34,193 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 357e17d9996b2cb73c13bcd478ff4f03: 2023-02-13 17:54:34,194 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03., storeName=357e17d9996b2cb73c13bcd478ff4f03/cf1, priority=9, startTime=1676310874142; duration=0sec 2023-02-13 17:54:34,194 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,195 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 357e17d9996b2cb73c13bcd478ff4f03/cf2 of 357e17d9996b2cb73c13bcd478ff4f03 into b8ac9b395fa744ea9b543c6e81e7a488(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:34,196 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 357e17d9996b2cb73c13bcd478ff4f03: 2023-02-13 17:54:34,196 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03., storeName=357e17d9996b2cb73c13bcd478ff4f03/cf2, priority=9, startTime=1676310874143; duration=0sec 2023-02-13 17:54:34,196 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:34,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46577] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:40778 deadline: 1676310934364, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd. is not online on jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:34,477 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:54:34,485 INFO [Listener at localhost/44157] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 17:54:34,486 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:54:34,504 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:34,506 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 17:54:34,507 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-13 17:54:34,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 17:54:34,511 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310874511"}]},"ts":"1676310874511"} 2023-02-13 17:54:34,513 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 17:54:34,515 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 17:54:34,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, UNASSIGN}] 2023-02-13 17:54:34,517 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, UNASSIGN 2023-02-13 17:54:34,518 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=abc074cb651d61f0e94ec5028574f5d0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:34,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874518"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310874518"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310874518"}]},"ts":"1676310874518"} 2023-02-13 17:54:34,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure abc074cb651d61f0e94ec5028574f5d0, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:34,522 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, UNASSIGN 2023-02-13 17:54:34,523 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=357e17d9996b2cb73c13bcd478ff4f03, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:34,523 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874523"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310874523"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310874523"}]},"ts":"1676310874523"} 2023-02-13 17:54:34,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 357e17d9996b2cb73c13bcd478ff4f03, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:34,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 17:54:34,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing abc074cb651d61f0e94ec5028574f5d0, disabling compactions & flushes 2023-02-13 17:54:34,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. after waiting 0 ms 2023-02-13 17:54:34,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,684 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-top] to archive 2023-02-13 17:54:34,685 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:34,686 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,688 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,689 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,690 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,691 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,693 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,694 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,696 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-top] to archive 2023-02-13 17:54:34,697 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:34,699 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,700 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,702 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,703 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,704 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,705 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,707 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:54:34,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:34,711 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0. 2023-02-13 17:54:34,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for abc074cb651d61f0e94ec5028574f5d0: Waiting for close lock at 1676310874671Running coprocessor pre-close hooks at 1676310874671Disabling compacts and flushes for region at 1676310874671Disabling writes for close at 1676310874672 (+1 ms)Writing region close event to WAL at 1676310874708 (+36 ms)Running coprocessor post-close hooks at 1676310874711 (+3 ms)Closed at 1676310874711 2023-02-13 17:54:34,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 357e17d9996b2cb73c13bcd478ff4f03, disabling compactions & flushes 2023-02-13 17:54:34,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. after waiting 0 ms 2023-02-13 17:54:34,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=abc074cb651d61f0e94ec5028574f5d0, regionState=CLOSED 2023-02-13 17:54:34,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874713"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310874713"}]},"ts":"1676310874713"} 2023-02-13 17:54:34,714 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3-bottom] to archive 2023-02-13 17:54:34,715 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:34,717 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 17:54:34,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure abc074cb651d61f0e94ec5028574f5d0, server=jenkins-hbase4.apache.org,46577,1676310860699 in 196 msec 2023-02-13 17:54:34,718 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,719 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=abc074cb651d61f0e94ec5028574f5d0, UNASSIGN in 202 msec 2023-02-13 17:54:34,720 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,721 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,722 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,723 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,725 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,726 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445-bottom] to archive 2023-02-13 17:54:34,727 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:34,728 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,729 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,731 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,732 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,733 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,734 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,735 DEBUG [StoreCloser-testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:54:34,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:34,739 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03. 2023-02-13 17:54:34,739 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 357e17d9996b2cb73c13bcd478ff4f03: Waiting for close lock at 1676310874713Running coprocessor pre-close hooks at 1676310874713Disabling compacts and flushes for region at 1676310874713Disabling writes for close at 1676310874713Writing region close event to WAL at 1676310874736 (+23 ms)Running coprocessor post-close hooks at 1676310874739 (+3 ms)Closed at 1676310874739 2023-02-13 17:54:34,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=357e17d9996b2cb73c13bcd478ff4f03, regionState=CLOSED 2023-02-13 17:54:34,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310874741"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310874741"}]},"ts":"1676310874741"} 2023-02-13 17:54:34,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 17:54:34,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 357e17d9996b2cb73c13bcd478ff4f03, server=jenkins-hbase4.apache.org,46577,1676310860699 in 217 msec 2023-02-13 17:54:34,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 17:54:34,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=357e17d9996b2cb73c13bcd478ff4f03, UNASSIGN in 228 msec 2023-02-13 17:54:34,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310874747"}]},"ts":"1676310874747"} 2023-02-13 17:54:34,749 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 17:54:34,750 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 17:54:34,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 244 msec 2023-02-13 17:54:34,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 17:54:34,814 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 17:54:34,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-13 17:54:34,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,817 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 17:54:34,824 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,824 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,824 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,827 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/recovered.edits] 2023-02-13 17:54:34,828 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/recovered.edits] 2023-02-13 17:54:34,828 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/recovered.edits] 2023-02-13 17:54:34,831 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/261aac4ccdb5486c8a2abcff6c27ae33 2023-02-13 17:54:34,832 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/.regioninfo to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/.regioninfo 2023-02-13 17:54:34,832 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/.regioninfo to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/.regioninfo 2023-02-13 17:54:34,833 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/3d97b33340814bd29f377b5733118cb3 2023-02-13 17:54:34,835 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/9aabdbee243043a68f9705662f14c2dd 2023-02-13 17:54:34,835 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,835 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/261aac4ccdb5486c8a2abcff6c27ae33.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,836 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/a099b9911d6e4a6d83cbab01040dd7ea 2023-02-13 17:54:34,837 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,837 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/3d97b33340814bd29f377b5733118cb3.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,838 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/cf514e1550df43efa7b560505c5d00aa 2023-02-13 17:54:34,839 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,839 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/9aabdbee243043a68f9705662f14c2dd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,839 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/e3289d3c05fc4fb9be58d0b91d1ead12 2023-02-13 17:54:34,840 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,840 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/a099b9911d6e4a6d83cbab01040dd7ea.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,841 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf1/f37abd41a14844f4a623af7a738e56ac 2023-02-13 17:54:34,842 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,842 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/cf514e1550df43efa7b560505c5d00aa.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,844 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,844 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/e3289d3c05fc4fb9be58d0b91d1ead12.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,845 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/0e5d7e738c004228a3e6640aa6bce8cd 2023-02-13 17:54:34,845 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,846 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf1/f37abd41a14844f4a623af7a738e56ac.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,846 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/56912a261d8d45b7ba1c92be93ed3c3e 2023-02-13 17:54:34,849 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/6bf46134703546ef88de9cb09ab0852f 2023-02-13 17:54:34,850 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,851 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/0e5d7e738c004228a3e6640aa6bce8cd.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,851 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/95e539f89f6f412f8c9d5c3b48d80c24 2023-02-13 17:54:34,853 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,853 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/b75ce08924064a9c9ab3ff086ea1e429 2023-02-13 17:54:34,854 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/56912a261d8d45b7ba1c92be93ed3c3e.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,855 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,855 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/d32e610bb609432295aae83523abc445 2023-02-13 17:54:34,856 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/6bf46134703546ef88de9cb09ab0852f.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,856 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,857 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/cf2/e6cd356d6cfc4e3593d0fa9e756bc865 2023-02-13 17:54:34,858 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,859 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/95e539f89f6f412f8c9d5c3b48d80c24.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,860 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/b75ce08924064a9c9ab3ff086ea1e429.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,861 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,862 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd/recovered.edits/85.seqid 2023-02-13 17:54:34,862 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/d32e610bb609432295aae83523abc445.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,863 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/abc074cb651d61f0e94ec5028574f5d0/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,863 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,864 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/357e17d9996b2cb73c13bcd478ff4f03/cf2/e6cd356d6cfc4e3593d0fa9e756bc865.eb1733af0943ef056fdc5749cdda62fd 2023-02-13 17:54:34,866 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/b126c6cc9d2d42eebfe23c9b15642ec4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf1/b126c6cc9d2d42eebfe23c9b15642ec4 2023-02-13 17:54:34,867 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/124dec46945a484b88a49761aa9087fc to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf1/124dec46945a484b88a49761aa9087fc 2023-02-13 17:54:34,869 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/2731c34c9785403e900774b4f7da89de to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/cf2/2731c34c9785403e900774b4f7da89de 2023-02-13 17:54:34,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b8ac9b395fa744ea9b543c6e81e7a488 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/cf2/b8ac9b395fa744ea9b543c6e81e7a488 2023-02-13 17:54:34,873 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0/recovered.edits/90.seqid 2023-02-13 17:54:34,873 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03/recovered.edits/90.seqid 2023-02-13 17:54:34,874 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:34,874 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitWithoutPONR/357e17d9996b2cb73c13bcd478ff4f03 2023-02-13 17:54:34,874 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 17:54:34,877 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,880 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 17:54:34,882 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 17:54:34,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,884 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 17:54:34,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310874884"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:34,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310874884"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:34,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310874884"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:34,886 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:54:34,886 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => eb1733af0943ef056fdc5749cdda62fd, NAME => 'testSplitWithoutPONR,,1676310868463.eb1733af0943ef056fdc5749cdda62fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 357e17d9996b2cb73c13bcd478ff4f03, NAME => 'testSplitWithoutPONR,,1676310870466.357e17d9996b2cb73c13bcd478ff4f03.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => abc074cb651d61f0e94ec5028574f5d0, NAME => 'testSplitWithoutPONR,41,1676310870466.abc074cb651d61f0e94ec5028574f5d0.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 17:54:34,887 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 17:54:34,887 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310874887"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:34,888 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 17:54:34,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:34,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 77 msec 2023-02-13 17:54:34,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 17:54:34,920 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 17:54:34,937 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=399 (was 391) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57624 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57652 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:45842 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54222 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:45866 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54296 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:45864 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54200 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:45874 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57842 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54250 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57608 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57872 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54280 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:54234 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1500651284_15 at /127.0.0.1:57634 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x26616879-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:46577-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=870 (was 845) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 283), ProcessCount=168 (was 168), AvailableMemoryMB=1159 (was 1294) 2023-02-13 17:54:34,951 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=399, OpenFileDescriptor=870, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=168, AvailableMemoryMB=1158 2023-02-13 17:54:34,951 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:34,954 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:34,954 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:34,956 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:34,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:34,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:34,960 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:34,961 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f empty. 2023-02-13 17:54:34,961 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:34,961 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 17:54:34,972 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:34,973 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => e0234a68997407cc17c3cb2323d2a02f, NAME => 'testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:34,981 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:34,981 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing e0234a68997407cc17c3cb2323d2a02f, disabling compactions & flushes 2023-02-13 17:54:34,981 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:34,981 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:34,981 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. after waiting 0 ms 2023-02-13 17:54:34,981 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:34,981 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:34,982 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for e0234a68997407cc17c3cb2323d2a02f: Waiting for close lock at 1676310874981Disabling compacts and flushes for region at 1676310874981Disabling writes for close at 1676310874981Writing region close event to WAL at 1676310874981Closed at 1676310874981 2023-02-13 17:54:34,984 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:34,985 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310874984"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310874984"}]},"ts":"1676310874984"} 2023-02-13 17:54:34,986 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:34,987 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:34,987 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310874987"}]},"ts":"1676310874987"} 2023-02-13 17:54:34,988 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 17:54:34,991 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:34,992 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:34,992 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:34,992 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:34,992 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:34,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN}] 2023-02-13 17:54:34,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:34,994 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:35,145 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:35,146 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310875146"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310875146"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310875146"}]},"ts":"1676310875146"} 2023-02-13 17:54:35,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:35,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e0234a68997407cc17c3cb2323d2a02f, NAME => 'testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:35,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:35,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:35,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,305 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,306 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1 2023-02-13 17:54:35,306 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1 2023-02-13 17:54:35,307 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0234a68997407cc17c3cb2323d2a02f columnFamilyName cf1 2023-02-13 17:54:35,307 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(310): Store=e0234a68997407cc17c3cb2323d2a02f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:35,307 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,308 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2 2023-02-13 17:54:35,308 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2 2023-02-13 17:54:35,309 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0234a68997407cc17c3cb2323d2a02f columnFamilyName cf2 2023-02-13 17:54:35,309 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(310): Store=e0234a68997407cc17c3cb2323d2a02f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:35,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:35,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:35,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:35,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e0234a68997407cc17c3cb2323d2a02f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73081602, jitterRate=0.08900073170661926}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:35,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e0234a68997407cc17c3cb2323d2a02f: Running coprocessor pre-open hook at 1676310875304Writing region info on filesystem at 1676310875304Initializing all the Stores at 1676310875304Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310875304Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310875305 (+1 ms)Cleaning up temporary data from old regions at 1676310875311 (+6 ms)Cleaning up detritus from prior splits at 1676310875311Running coprocessor post-open hooks at 1676310875315 (+4 ms)Region opened successfully at 1676310875315 2023-02-13 17:54:35,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f., pid=43, masterSystemTime=1676310875299 2023-02-13 17:54:35,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,318 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310875318"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310875318"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310875318"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310875318"}]},"ts":"1676310875318"} 2023-02-13 17:54:35,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 17:54:35,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 in 172 msec 2023-02-13 17:54:35,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 17:54:35,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN in 330 msec 2023-02-13 17:54:35,325 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:35,325 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310875325"}]},"ts":"1676310875325"} 2023-02-13 17:54:35,326 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 17:54:35,332 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:35,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 377 msec 2023-02-13 17:54:35,466 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:35,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:35,468 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,468 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,468 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:35,468 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:35,468 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,468 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,469 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,469 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,469 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,469 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,470 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,470 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,471 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:35,472 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:35,472 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,472 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:35,473 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:54:35,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,475 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,476 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,476 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,494 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/2faf5df9f4824feaad0def32aa6c720e 2023-02-13 17:54:35,511 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/6d7c86d5b7c84f89b3001804f422e682 2023-02-13 17:54:35,516 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/2faf5df9f4824feaad0def32aa6c720e as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e 2023-02-13 17:54:35,520 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:35,521 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/6d7c86d5b7c84f89b3001804f422e682 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 2023-02-13 17:54:35,526 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:35,527 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 54ms, sequenceid=6, compaction requested=false 2023-02-13 17:54:35,527 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,527 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:35,527 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:35,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:35,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,528 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,531 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,531 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:35,534 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:35,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e8939f6[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:35,534 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,536 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,536 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,536 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:35,536 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,536 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:35,536 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,537 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:35,537 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,537 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,537 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:35,538 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,538 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,538 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,539 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,539 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,539 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,540 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,540 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,541 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,541 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:35,544 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:35,544 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:35,544 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:35,544 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:35,544 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:35,544 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,547 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,546 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,547 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,548 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,549 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,549 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,550 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,553 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,553 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,553 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:35,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,552 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,553 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:35,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,654 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:35,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:35,669 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:35,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:35,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:35,670 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,670 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,671 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:35,671 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:35,671 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,671 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,672 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,672 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,672 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,672 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,672 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,672 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:35,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,674 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,674 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,674 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,674 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,675 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,675 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:35,675 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:35,675 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,675 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:35,676 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:35,677 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,678 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,679 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,679 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,679 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,688 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/18e82713df5e4d6eb26ecdc0dba10f15 2023-02-13 17:54:35,701 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/787aee2d8aef42b78f7fc1103a9dd285 2023-02-13 17:54:35,706 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/18e82713df5e4d6eb26ecdc0dba10f15 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 2023-02-13 17:54:35,710 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:35,711 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/787aee2d8aef42b78f7fc1103a9dd285 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 2023-02-13 17:54:35,715 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:35,716 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 40ms, sequenceid=19, compaction requested=false 2023-02-13 17:54:35,716 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:35,716 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,716 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,718 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,718 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,718 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,718 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:35,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bd68eec[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:35,720 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:35,720 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,721 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:35,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:35,721 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:35,723 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,723 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,724 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,724 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,727 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:35,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:35,730 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:35,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:35,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:35,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:35,730 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,731 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,732 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,732 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,733 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,734 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,734 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,735 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,739 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,740 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:35,739 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,740 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:35,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,739 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,740 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:35,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,740 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,740 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,741 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,840 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:35,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:35,855 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:35,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:35,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:35,856 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,856 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,856 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:35,856 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:35,857 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,857 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,858 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,858 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,858 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,858 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,859 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,860 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,860 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:35,861 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:35,861 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,861 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,861 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:35,861 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:35,861 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,861 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:35,862 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,863 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,872 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/2f4a676b053e47d0b0996a57371e17f4 2023-02-13 17:54:35,885 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/21d46fd22fe4437887c84b875224c3d0 2023-02-13 17:54:35,889 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/2f4a676b053e47d0b0996a57371e17f4 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 2023-02-13 17:54:35,893 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:35,894 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/21d46fd22fe4437887c84b875224c3d0 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 2023-02-13 17:54:35,897 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:35,898 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 37ms, sequenceid=32, compaction requested=true 2023-02-13 17:54:35,898 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:35,898 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:35,898 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:35,899 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:35,901 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,901 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,901 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,901 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:35,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:35,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a60d9cc[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:35,903 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:35,903 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,905 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,905 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:35,905 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,906 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,906 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,907 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:35,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:35,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:35,910 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:35,910 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:35,910 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:35,910 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:35,910 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,913 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,913 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,914 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,914 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:35,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:35,916 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:35,916 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,917 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:35,923 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:35,923 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,923 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:35,922 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:35,923 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:35,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,024 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:36,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:36,044 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:36,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:36,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:36,046 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:36,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:36,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,046 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,048 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,048 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,048 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,051 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,050 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,051 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,051 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,051 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:36,051 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:36,051 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,051 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:36,051 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:36,053 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,053 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,053 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,054 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,054 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,054 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,067 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/6e62a549c5734c068fe7fc9c74c2d24d 2023-02-13 17:54:36,079 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 2023-02-13 17:54:36,083 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/6e62a549c5734c068fe7fc9c74c2d24d as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d 2023-02-13 17:54:36,087 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:36,087 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 2023-02-13 17:54:36,092 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:36,093 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 42ms, sequenceid=45, compaction requested=true 2023-02-13 17:54:36,093 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:36,093 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,093 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:36,094 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:36,094 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,094 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,094 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,095 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:36,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a780e7c[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:36,097 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:36,097 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,099 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,099 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:36,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:36,099 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,101 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,103 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,103 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,103 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,103 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:36,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:36,106 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:36,106 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:36,106 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:36,106 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:36,106 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,108 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,108 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,108 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,109 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,110 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,110 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,111 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,114 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,114 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:36,114 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,114 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:36,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,115 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,215 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:36,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:36,233 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:36,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:36,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:36,235 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,235 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,235 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:36,235 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:36,235 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,236 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,237 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,237 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,237 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:36,239 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:36,240 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,240 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:36,240 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:36,241 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,242 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,242 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,242 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,242 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,242 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,251 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/612635170a5b4a53a21e615edc2165af 2023-02-13 17:54:36,265 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/8003a8ef0e334bfd93188fca767d4379 2023-02-13 17:54:36,270 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/612635170a5b4a53a21e615edc2165af as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af 2023-02-13 17:54:36,274 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:36,274 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/8003a8ef0e334bfd93188fca767d4379 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 2023-02-13 17:54:36,278 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:36,279 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 39ms, sequenceid=58, compaction requested=true 2023-02-13 17:54:36,280 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:36,280 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,280 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,286 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:36,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6db9dc4c[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:36,288 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:36,289 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,290 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,290 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:36,290 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:36,291 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,291 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:36,292 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,297 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,297 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,297 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,298 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:36,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:36,301 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:36,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:36,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:36,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:36,301 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,302 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,302 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,303 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,303 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,304 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,305 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,306 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,306 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,308 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,308 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:36,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,309 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,409 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:36,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:36,425 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:36,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:36,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 17:54:36,426 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,427 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,427 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 17:54:36,427 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:54:36,427 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,427 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,428 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,428 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,428 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,428 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,429 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,429 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:36,430 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:36,431 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:36,431 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,431 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. started... 2023-02-13 17:54:36,431 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:36,433 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,434 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,435 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,435 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,440 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/3035dec1800c42f4b48615024343284c 2023-02-13 17:54:36,459 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/413abf2362964c51b4875b7321b03504 2023-02-13 17:54:36,463 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/3035dec1800c42f4b48615024343284c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c 2023-02-13 17:54:36,468 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:36,468 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/413abf2362964c51b4875b7321b03504 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 2023-02-13 17:54:36,473 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:36,475 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 44ms, sequenceid=71, compaction requested=true 2023-02-13 17:54:36,475 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:36,475 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:36,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:36,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:36,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 17:54:36,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 17:54:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,477 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 17:54:36,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2779ef9[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:36,479 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:54:36,479 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,480 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:36,481 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,483 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,484 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 17:54:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,484 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 17:54:36,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:54:36,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:54:36,487 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 17:54:36,487 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:36,487 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:36,487 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 17:54:36,487 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,489 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:36,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,490 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,490 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,492 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:36,492 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 17:54:36,492 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 17:54:36,504 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:36,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:36,505 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:36,504 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 17:54:36,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,505 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:36,505 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,505 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:36,506 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:36,506 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 17:54:36,506 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 17:54:36,605 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:36,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:36,866 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 17:54:36,867 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:36,868 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:36,870 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:36,870 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:36,870 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:36,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:36,870 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:36,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:36,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:37,119 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:37,119 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:37,120 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:37,121 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 17:54:37,121 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:37,122 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:37,122 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:37,122 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:37,122 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,123 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:37,123 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,123 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,123 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,124 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:37,124 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,124 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,124 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:37,125 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,125 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:37,125 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,125 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,125 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,125 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:37,126 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:37,126 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:37,127 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310907126 2023-02-13 17:54:37,128 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,128 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:37,130 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,131 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,131 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:37,131 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310937131 2023-02-13 17:54:37,131 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310997131 2023-02-13 17:54:37,131 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:37,131 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:37,136 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:37,136 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:37,136 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:37,381 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,381 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:37,381 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:37,381 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:37,383 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:37,383 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:37,383 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:37,383 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:37,383 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:37,384 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:37,385 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 17:54:37,385 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:37,385 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:37,386 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:37,386 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:37,386 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,386 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:37,386 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,387 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,387 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,387 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:37,387 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,387 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,388 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:37,388 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,388 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:37,388 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,388 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,388 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,388 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:37,389 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:37,389 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:37,390 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310907390 2023-02-13 17:54:37,391 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,391 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:37,394 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,394 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,394 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:37,394 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310937394 2023-02-13 17:54:37,394 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310997394 2023-02-13 17:54:37,394 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:37,395 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:37,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN}] 2023-02-13 17:54:37,397 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:37,397 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:37,397 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:37,645 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:37,645 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:37,646 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:37,646 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:37,647 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 17:54:37,648 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:37,648 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:37,649 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:37,649 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:37,649 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,650 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:37,650 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,650 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,651 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,651 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:37,651 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,652 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,652 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:37,653 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,653 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:37,653 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,653 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,653 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,653 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:54:37,654 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:37,654 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:37,655 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310907655 2023-02-13 17:54:37,656 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:37,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,659 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,659 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,659 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:37,660 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310937659 2023-02-13 17:54:37,660 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310997660 2023-02-13 17:54:37,660 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:37,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN}] 2023-02-13 17:54:37,660 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:37,662 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:37,663 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:37,663 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310877663"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310877663"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310877663"}]},"ts":"1676310877663"} 2023-02-13 17:54:37,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:37,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:37,666 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:37,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:37,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:37,910 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,910 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:37,910 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:37,910 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:37,913 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:37,913 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:37,913 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:37,913 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:37,913 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:37,914 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:37,914 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:37,915 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:37,915 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:37,915 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:37,915 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:37,916 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:37,916 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,916 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:37,916 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,917 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:37,917 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,917 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:37,917 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,917 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:37,918 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:37,918 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,918 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:37,918 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:37,919 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,919 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,919 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:37,919 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:37,919 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:37,919 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:37,919 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:37,920 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:37,920 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:37,920 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310907920 2023-02-13 17:54:37,921 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:37,921 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:37,924 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,924 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:37,924 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:37,924 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310937924 2023-02-13 17:54:37,924 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310997924 2023-02-13 17:54:37,924 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:37,924 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:37,925 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310877924"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310877924"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310877924"}]},"ts":"1676310877924"} 2023-02-13 17:54:37,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:37,926 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:38,079 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e0234a68997407cc17c3cb2323d2a02f, disabling compactions & flushes 2023-02-13 17:54:38,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:38,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:38,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. after waiting 0 ms 2023-02-13 17:54:38,080 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:38,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e0234a68997407cc17c3cb2323d2a02f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:54:38,090 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/7fc2f5721c874cddbf6f0c58e939d489 2023-02-13 17:54:38,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/6ffbf2a07e984ea7a64dd2d94434e79c 2023-02-13 17:54:38,108 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/7fc2f5721c874cddbf6f0c58e939d489 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 2023-02-13 17:54:38,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:38,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/6ffbf2a07e984ea7a64dd2d94434e79c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c 2023-02-13 17:54:38,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:54:38,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 37ms, sequenceid=82, compaction requested=true 2023-02-13 17:54:38,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:54:38,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:38,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:38,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e0234a68997407cc17c3cb2323d2a02f: Waiting for close lock at 1676310878079Running coprocessor pre-close hooks at 1676310878080 (+1 ms)Disabling compacts and flushes for region at 1676310878080Disabling writes for close at 1676310878080Obtaining lock to block concurrent updates at 1676310878080Preparing flush snapshotting stores in e0234a68997407cc17c3cb2323d2a02f at 1676310878080Finished memstore snapshotting testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310878080Flushing stores of testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. at 1676310878081 (+1 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf1: creating writer at 1676310878082 (+1 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf1: appending metadata at 1676310878083 (+1 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf1: closing flushed file at 1676310878083Flushing e0234a68997407cc17c3cb2323d2a02f/cf2: creating writer at 1676310878095 (+12 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf2: appending metadata at 1676310878096 (+1 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf2: closing flushed file at 1676310878096Flushing e0234a68997407cc17c3cb2323d2a02f/cf1: reopening flushed file at 1676310878109 (+13 ms)Flushing e0234a68997407cc17c3cb2323d2a02f/cf2: reopening flushed file at 1676310878113 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e0234a68997407cc17c3cb2323d2a02f in 37ms, sequenceid=82, compaction requested=true at 1676310878117 (+4 ms)Writing region close event to WAL at 1676310878119 (+2 ms)Running coprocessor post-close hooks at 1676310878121 (+2 ms)Closed at 1676310878122 (+1 ms) 2023-02-13 17:54:38,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED 2023-02-13 17:54:38,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310878124"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310878124"}]},"ts":"1676310878124"} 2023-02-13 17:54:38,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:38,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:38,126 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:38,174 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:38,175 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:38,175 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:38,176 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:38,177 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:38,177 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 17:54:38,177 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 17:54:38,177 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:38,178 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:38,178 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:38,178 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:38,178 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,179 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:38,179 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,179 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,180 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,180 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:38,180 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,180 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,180 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:38,181 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,181 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,181 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:38,181 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,181 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:38,182 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:38,182 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:38,183 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:38,183 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:38,188 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310908188 2023-02-13 17:54:38,189 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:38,189 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,192 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,192 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,192 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:38,192 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310938192 2023-02-13 17:54:38,192 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310998192 2023-02-13 17:54:38,192 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:38,192 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310878192"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310878192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310878192"}]},"ts":"1676310878192"} 2023-02-13 17:54:38,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:38,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 17:54:38,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 in 268 msec 2023-02-13 17:54:38,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:38,197 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:38,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:38,442 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,443 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:38,443 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:38,443 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:38,444 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:38,444 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:38,444 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:38,444 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:38,444 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:38,445 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:38,446 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:38,446 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 17:54:38,446 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:38,446 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:38,447 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:38,447 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:38,447 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:38,447 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,448 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:38,448 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,448 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,448 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,449 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:38,449 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,449 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,449 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:38,450 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,450 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,450 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,450 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:38,450 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,451 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,451 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,451 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,451 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,451 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:38,451 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:38,452 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:38,452 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:38,452 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310908452 2023-02-13 17:54:38,452 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:38,453 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,455 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,455 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,455 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:38,455 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310938455 2023-02-13 17:54:38,456 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310998456 2023-02-13 17:54:38,456 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:38,456 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,458 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:38,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 17:54:38,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN in 798 msec 2023-02-13 17:54:38,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:38,467 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:38,468 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:38,706 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:38,706 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:38,707 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:38,707 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:38,707 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:38,708 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:38,708 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:38,709 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:38,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:38,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:38,710 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:38,710 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:38,710 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,711 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:38,711 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,711 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,712 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,712 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:38,712 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,712 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,713 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,714 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:54:38,714 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:38,715 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:38,715 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310908715 2023-02-13 17:54:38,716 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,716 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:38,719 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,719 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,719 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:38,719 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310938719 2023-02-13 17:54:38,719 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310998719 2023-02-13 17:54:38,719 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:38,719 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,719 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,719 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,724 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:38,733 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=e0234a68997407cc17c3cb2323d2a02f, threads=14 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,734 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,735 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,735 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,736 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,737 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,840 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,843 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,845 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,848 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,850 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,869 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,876 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,883 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,884 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,888 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,888 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,890 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,892 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,896 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,896 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region e0234a68997407cc17c3cb2323d2a02f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:38,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:38,932 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:38,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:38,969 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,969 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:38,970 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:38,970 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:38,971 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:38,972 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 17:54:38,972 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:38,972 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:38,972 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:38,973 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:38,973 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:38,973 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:38,974 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,974 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:38,974 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,975 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:38,975 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,975 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:38,976 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,976 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:38,976 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:38,976 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,976 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:38,977 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:38,977 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:38,977 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,977 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,977 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,977 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,977 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:38,977 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:54:38,978 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:38,978 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:38,978 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310908978 2023-02-13 17:54:38,979 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:38,980 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:38,982 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,982 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:38,982 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:38,982 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310938982 2023-02-13 17:54:38,982 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310998982 2023-02-13 17:54:38,982 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:38,991 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=e0234a68997407cc17c3cb2323d2a02f, threads=14 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,992 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,993 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:38,996 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,000 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,078 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,083 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,099 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,110 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,119 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,126 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,135 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,140 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,148 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,149 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,151 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,152 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,152 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,153 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 for region: e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:39,154 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region e0234a68997407cc17c3cb2323d2a02f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:39,191 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:39,197 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/3f75ed0ecbae6a34d42f633446ce9c87/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:39,198 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/8eafaada8ae1d0fb7136ae4ebf3eaa0e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:54:39,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:39,198 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:39,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:39,233 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,233 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:39,233 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:39,233 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:39,234 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:39,234 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:39,234 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:39,234 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:39,234 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:39,235 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:39,236 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 17:54:39,236 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:39,236 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:39,236 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:39,237 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:39,237 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:39,237 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:39,238 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,238 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:39,238 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,238 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,239 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,239 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:39,239 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,239 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,240 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:39,240 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,241 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:39,241 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:39,243 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:39,243 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,243 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,243 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,243 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,243 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,243 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 17:54:39,244 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:39,244 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:39,246 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310909246 2023-02-13 17:54:39,248 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:39,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,250 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,250 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,250 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:39,250 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310939250 2023-02-13 17:54:39,251 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310999251 2023-02-13 17:54:39,251 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:39,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:39,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:39,257 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:39,257 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:39,435 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:54:39,501 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,501 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:39,502 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:39,502 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:39,503 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:39,504 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 17:54:39,504 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:39,504 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:39,504 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:39,505 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:39,505 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:39,505 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:39,505 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:39,506 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:39,507 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:39,507 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,508 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,508 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,508 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,508 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,508 DEBUG [Listener at localhost/44157] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,508 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:39,508 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:39,509 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:39,509 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310909509 2023-02-13 17:54:39,512 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:39,512 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,514 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,514 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,514 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:39,514 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310939514 2023-02-13 17:54:39,514 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310999514 2023-02-13 17:54:39,514 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:39,515 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:39,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:39,526 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:39,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:39,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:39,765 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,766 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:39,766 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:39,766 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:39,766 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:39,766 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:39,767 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:39,767 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:39,767 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:39,767 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:39,768 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 17:54:39,768 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:39,768 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:39,769 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:39,769 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:39,769 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:39,769 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,770 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:39,771 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,771 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:39,771 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:39,772 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,772 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:39,772 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:39,772 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:39,772 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,772 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,773 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,773 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,773 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:39,773 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:39,773 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:39,774 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:39,774 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310909774 2023-02-13 17:54:39,776 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:39,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:39,779 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,779 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:39,779 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:39,779 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310939779 2023-02-13 17:54:39,779 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676310999779 2023-02-13 17:54:39,779 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:39,779 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:39,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:39,781 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:39,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:39,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:40,030 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,030 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:40,030 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:40,030 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:40,031 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:40,031 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:40,031 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:40,031 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:40,031 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:40,032 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:40,032 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 17:54:40,033 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:40,033 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:40,033 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:40,033 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:40,033 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:40,034 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:40,034 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,034 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:40,034 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,034 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,035 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,035 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:40,035 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,035 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,035 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:40,036 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,036 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:40,036 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:40,036 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:40,036 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,036 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,036 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,036 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,037 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,037 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:40,037 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:40,038 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:40,038 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310910038 2023-02-13 17:54:40,039 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:40,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,042 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,042 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,043 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:40,043 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310940043 2023-02-13 17:54:40,043 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311000043 2023-02-13 17:54:40,043 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:40,043 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:40,045 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:40,045 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:40,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:40,045 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:40,293 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,293 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:40,293 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:40,293 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:40,294 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:40,295 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:40,295 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:40,295 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:40,295 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:40,295 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:40,296 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 17:54:40,296 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:40,296 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:40,297 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:40,297 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:40,297 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:40,297 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:40,297 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:40,298 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:40,299 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:40,299 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,300 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,300 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,300 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,300 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,300 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:40,300 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:40,301 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:40,301 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310910301 2023-02-13 17:54:40,301 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:40,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,305 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,305 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,305 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:40,305 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310940305 2023-02-13 17:54:40,305 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311000305 2023-02-13 17:54:40,305 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:54:40,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:40,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN] 2023-02-13 17:54:40,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:40,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:40,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:40,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:40,555 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:40,556 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:40,556 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:40,557 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:40,558 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,559 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,560 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,560 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:40,560 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,560 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,560 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:40,561 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,561 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:40,561 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:40,561 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:40,561 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:40,562 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,562 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,562 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,562 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,562 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,562 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,562 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 17:54:40,562 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:40,563 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:40,563 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:40,563 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310910563 2023-02-13 17:54:40,563 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:40,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,566 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:40,568 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,568 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,568 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:40,568 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310940568 2023-02-13 17:54:40,568 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311000568 2023-02-13 17:54:40,568 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:54:40,568 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:40,573 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:40,573 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:40,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:40,573 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:40,818 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:40,819 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:40,819 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:40,820 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:40,820 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:54:40,821 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:40,821 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:40,821 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:40,821 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:40,821 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,822 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:40,823 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:40,823 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,823 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,824 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,824 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:40,824 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 17:54:40,824 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:40,825 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:40,825 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:40,825 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310910825 2023-02-13 17:54:40,827 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:40,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:40,828 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:40,830 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,830 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:40,830 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:40,830 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310940830 2023-02-13 17:54:40,830 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311000830 2023-02-13 17:54:40,830 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:54:40,830 INFO [PEWorker-1] procedure.MasterProcedureScheduler(718): Waiting on xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN held by pid=44 2023-02-13 17:54:40,830 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:40,830 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:40,832 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:40,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:40,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:40,832 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:41,080 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,080 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:41,081 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:41,081 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:41,082 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:41,082 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:41,082 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:41,082 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:41,082 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:41,083 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:41,083 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 17:54:41,083 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:41,084 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:41,084 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:41,084 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:41,084 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:41,084 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:41,085 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:41,086 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,086 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,086 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,086 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:41,087 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 17:54:41,087 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:41,087 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:41,088 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:41,088 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310911088 2023-02-13 17:54:41,088 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:41,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,091 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,093 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,093 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,093 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:41,093 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310941093 2023-02-13 17:54:41,094 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311001094 2023-02-13 17:54:41,094 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 17:54:41,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:41,096 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:41,096 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 17:54:41,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:41,096 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:41,344 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,344 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:41,344 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:41,344 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:41,346 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:41,347 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:41,347 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:41,347 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:41,347 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:41,347 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:41,348 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 17:54:41,348 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 17:54:41,349 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:41,349 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:41,349 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:41,349 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:41,349 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,350 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:41,350 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,350 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,350 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,350 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:41,351 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,351 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,351 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:41,352 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,352 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,352 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:41,352 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,352 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,352 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,353 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:54:41,353 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 17:54:41,353 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:41,354 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:41,354 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:41,354 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310911354 2023-02-13 17:54:41,355 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:41,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,358 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,360 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,360 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,360 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:41,360 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310941360 2023-02-13 17:54:41,360 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311001360 2023-02-13 17:54:41,360 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:54:41,360 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:54:41,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e exec-time=4.4930 sec 2023-02-13 17:54:41,363 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,363 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=true 2023-02-13 17:54:41,513 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:41,514 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:41,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310881514"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310881514"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310881514"}]},"ts":"1676310881514"} 2023-02-13 17:54:41,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:41,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:54:41,517 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:54:41,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:54:41,610 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:41,611 WARN [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:54:41,611 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:54:41,611 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:54:41,612 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:54:41,613 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 17:54:41,613 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:54:41,613 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:54:41,613 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:54:41,613 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:54:41,614 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,614 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:54:41,614 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,614 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:54:41,614 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,615 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:54:41,616 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,616 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 17:54:41,616 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:54:41,617 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:54:41,617 INFO [Listener at localhost/44157] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f to restore RIT 2023-02-13 17:54:41,617 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310911617 2023-02-13 17:54:41,618 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:54:41,618 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:54:41,620 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,620 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=e0234a68997407cc17c3cb2323d2a02f, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:54:41,620 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:54:41,620 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676310941620 2023-02-13 17:54:41,620 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311001620 2023-02-13 17:54:41,620 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:54:41,620 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:41,621 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310881620"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310881620"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310881620"}]},"ts":"1676310881620"} 2023-02-13 17:54:41,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:41,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:41,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e0234a68997407cc17c3cb2323d2a02f, NAME => 'testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:41,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:41,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:41,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,780 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,781 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1 2023-02-13 17:54:41,781 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1 2023-02-13 17:54:41,781 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0234a68997407cc17c3cb2323d2a02f columnFamilyName cf1 2023-02-13 17:54:41,791 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 2023-02-13 17:54:41,795 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 2023-02-13 17:54:41,798 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e 2023-02-13 17:54:41,801 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c 2023-02-13 17:54:41,804 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af 2023-02-13 17:54:41,808 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d 2023-02-13 17:54:41,811 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 2023-02-13 17:54:41,811 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(310): Store=e0234a68997407cc17c3cb2323d2a02f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:41,811 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,812 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2 2023-02-13 17:54:41,812 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2 2023-02-13 17:54:41,813 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0234a68997407cc17c3cb2323d2a02f columnFamilyName cf2 2023-02-13 17:54:41,819 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 2023-02-13 17:54:41,822 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 2023-02-13 17:54:41,827 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 2023-02-13 17:54:41,830 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c 2023-02-13 17:54:41,833 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 2023-02-13 17:54:41,837 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 2023-02-13 17:54:41,840 DEBUG [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 2023-02-13 17:54:41,840 INFO [StoreOpener-e0234a68997407cc17c3cb2323d2a02f-1] regionserver.HStore(310): Store=e0234a68997407cc17c3cb2323d2a02f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:41,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.splits 2023-02-13 17:54:41,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:41,845 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:41,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e0234a68997407cc17c3cb2323d2a02f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71159401, jitterRate=0.060357704758644104}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:41,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e0234a68997407cc17c3cb2323d2a02f: Running coprocessor pre-open hook at 1676310881778Writing region info on filesystem at 1676310881778Initializing all the Stores at 1676310881779 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310881779Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310881780 (+1 ms)Cleaning up temporary data from old regions at 1676310881842 (+62 ms)Cleaning up detritus from prior splits at 1676310881843 (+1 ms)Running coprocessor post-open hooks at 1676310881846 (+3 ms)Region opened successfully at 1676310881846 2023-02-13 17:54:41,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f., pid=48, masterSystemTime=1676310881774 2023-02-13 17:54:41,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:41,848 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:41,848 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:41,848 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:41,849 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 17:54:41,849 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 17:54:41,849 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1912): e0234a68997407cc17c3cb2323d2a02f/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:41,849 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1912): e0234a68997407cc17c3cb2323d2a02f/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:41,849 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e0234a68997407cc17c3cb2323d2a02f/cf2 in testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:41,849 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e0234a68997407cc17c3cb2323d2a02f/cf1 in testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:41,849 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp, totalSize=34.9 K 2023-02-13 17:54:41,849 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp, totalSize=34.9 K 2023-02-13 17:54:41,851 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 2faf5df9f4824feaad0def32aa6c720e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310875459 2023-02-13 17:54:41,851 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 6d7c86d5b7c84f89b3001804f422e682, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310875459 2023-02-13 17:54:41,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:41,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:41,851 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 18e82713df5e4d6eb26ecdc0dba10f15, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310875655 2023-02-13 17:54:41,851 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 787aee2d8aef42b78f7fc1103a9dd285, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310875655 2023-02-13 17:54:41,851 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:41,851 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 2f4a676b053e47d0b0996a57371e17f4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310875842 2023-02-13 17:54:41,851 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310881851"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310881851"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310881851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310881851"}]},"ts":"1676310881851"} 2023-02-13 17:54:41,852 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 21d46fd22fe4437887c84b875224c3d0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310875842 2023-02-13 17:54:41,852 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 6e62a549c5734c068fe7fc9c74c2d24d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310876026 2023-02-13 17:54:41,852 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting ba35be1a6a574dfdbfe706ed1c2f2c44, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310876026 2023-02-13 17:54:41,852 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 612635170a5b4a53a21e615edc2165af, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310876217 2023-02-13 17:54:41,853 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 8003a8ef0e334bfd93188fca767d4379, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310876217 2023-02-13 17:54:41,853 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 3035dec1800c42f4b48615024343284c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310876411 2023-02-13 17:54:41,853 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 413abf2362964c51b4875b7321b03504, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310876411 2023-02-13 17:54:41,853 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting 7fc2f5721c874cddbf6f0c58e939d489, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310876607 2023-02-13 17:54:41,853 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting 6ffbf2a07e984ea7a64dd2d94434e79c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310876607 2023-02-13 17:54:41,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 17:54:41,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 in 231 msec 2023-02-13 17:54:41,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN in 1.5480 sec 2023-02-13 17:54:41,871 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 17:54:41,873 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] throttle.PressureAwareThroughputController(145): e0234a68997407cc17c3cb2323d2a02f#cf1#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:41,873 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] throttle.PressureAwareThroughputController(145): e0234a68997407cc17c3cb2323d2a02f#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:41,874 DEBUG [Listener at localhost/44157] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 17:54:41,876 INFO [Listener at localhost/44157] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 17:54:41,882 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 17:54:41,883 INFO [Listener at localhost/44157] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 17:54:41,883 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 17:54:41,899 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf2/1ca06d17010c4f5cb0f1f99c927acc79 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/1ca06d17010c4f5cb0f1f99c927acc79 2023-02-13 17:54:41,906 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e0234a68997407cc17c3cb2323d2a02f/cf2 of e0234a68997407cc17c3cb2323d2a02f into 1ca06d17010c4f5cb0f1f99c927acc79(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:41,906 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:41,906 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f., storeName=e0234a68997407cc17c3cb2323d2a02f/cf2, priority=9, startTime=1676310881848; duration=0sec 2023-02-13 17:54:41,907 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:41,924 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:41,925 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 17:54:41,926 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-13 17:54:41,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:41,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 17:54:41,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310881929"}]},"ts":"1676310881929"} 2023-02-13 17:54:41,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 17:54:41,932 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 17:54:41,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN}] 2023-02-13 17:54:41,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN 2023-02-13 17:54:41,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:41,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310881934"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310881934"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310881934"}]},"ts":"1676310881934"} 2023-02-13 17:54:41,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:42,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 17:54:42,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e0234a68997407cc17c3cb2323d2a02f, disabling compactions & flushes 2023-02-13 17:54:42,087 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:42,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 17:54:42,298 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/.tmp/cf1/d84e8cca41c5453cbc439f6f7686c368 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/d84e8cca41c5453cbc439f6f7686c368 2023-02-13 17:54:42,303 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e0234a68997407cc17c3cb2323d2a02f/cf1 of e0234a68997407cc17c3cb2323d2a02f into d84e8cca41c5453cbc439f6f7686c368(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:42,303 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e0234a68997407cc17c3cb2323d2a02f: 2023-02-13 17:54:42,303 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:42,303 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f., storeName=e0234a68997407cc17c3cb2323d2a02f/cf1, priority=9, startTime=1676310881847; duration=0sec 2023-02-13 17:54:42,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:42,303 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:42,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. after waiting 0 ms 2023-02-13 17:54:42,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:42,303 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489] to archive 2023-02-13 17:54:42,304 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:42,306 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2faf5df9f4824feaad0def32aa6c720e 2023-02-13 17:54:42,307 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/18e82713df5e4d6eb26ecdc0dba10f15 2023-02-13 17:54:42,308 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/2f4a676b053e47d0b0996a57371e17f4 2023-02-13 17:54:42,309 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/6e62a549c5734c068fe7fc9c74c2d24d 2023-02-13 17:54:42,310 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/612635170a5b4a53a21e615edc2165af 2023-02-13 17:54:42,311 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/3035dec1800c42f4b48615024343284c 2023-02-13 17:54:42,313 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/7fc2f5721c874cddbf6f0c58e939d489 2023-02-13 17:54:42,318 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c] to archive 2023-02-13 17:54:42,318 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:42,319 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6d7c86d5b7c84f89b3001804f422e682 2023-02-13 17:54:42,320 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/787aee2d8aef42b78f7fc1103a9dd285 2023-02-13 17:54:42,321 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/21d46fd22fe4437887c84b875224c3d0 2023-02-13 17:54:42,322 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/ba35be1a6a574dfdbfe706ed1c2f2c44 2023-02-13 17:54:42,323 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/8003a8ef0e334bfd93188fca767d4379 2023-02-13 17:54:42,324 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/413abf2362964c51b4875b7321b03504 2023-02-13 17:54:42,325 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/6ffbf2a07e984ea7a64dd2d94434e79c 2023-02-13 17:54:42,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:54:42,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:42,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f. 2023-02-13 17:54:42,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e0234a68997407cc17c3cb2323d2a02f: Waiting for close lock at 1676310882087Running coprocessor pre-close hooks at 1676310882087Disabling compacts and flushes for region at 1676310882087Disabling writes for close at 1676310882303 (+216 ms)Writing region close event to WAL at 1676310882326 (+23 ms)Running coprocessor post-close hooks at 1676310882329 (+3 ms)Closed at 1676310882329 2023-02-13 17:54:42,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:42,331 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=e0234a68997407cc17c3cb2323d2a02f, regionState=CLOSED 2023-02-13 17:54:42,331 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310882331"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310882331"}]},"ts":"1676310882331"} 2023-02-13 17:54:42,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 17:54:42,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure e0234a68997407cc17c3cb2323d2a02f, server=jenkins-hbase4.apache.org,46577,1676310860699 in 398 msec 2023-02-13 17:54:42,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 17:54:42,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, UNASSIGN in 403 msec 2023-02-13 17:54:42,337 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310882337"}]},"ts":"1676310882337"} 2023-02-13 17:54:42,338 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 17:54:42,339 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 17:54:42,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 414 msec 2023-02-13 17:54:42,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 17:54:42,532 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 17:54:42,533 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-13 17:54:42,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,537 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 17:54:42,540 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:42,542 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits] 2023-02-13 17:54:42,544 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/d84e8cca41c5453cbc439f6f7686c368 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf1/d84e8cca41c5453cbc439f6f7686c368 2023-02-13 17:54:42,546 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/1ca06d17010c4f5cb0f1f99c927acc79 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/cf2/1ca06d17010c4f5cb0f1f99c927acc79 2023-02-13 17:54:42,549 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f/recovered.edits/90.seqid 2023-02-13 17:54:42,549 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackAndDoubleExecution/e0234a68997407cc17c3cb2323d2a02f 2023-02-13 17:54:42,550 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 17:54:42,552 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,554 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 17:54:42,556 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 17:54:42,557 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,557 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 17:54:42,557 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310882557"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:42,558 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 17:54:42,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e0234a68997407cc17c3cb2323d2a02f, NAME => 'testRollbackAndDoubleExecution,,1676310874955.e0234a68997407cc17c3cb2323d2a02f.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 17:54:42,558 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 17:54:42,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310882558"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:42,560 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 17:54:42,562 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:54:42,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-02-13 17:54:42,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 17:54:42,638 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 17:54:42,652 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395 (was 399), OpenFileDescriptor=859 (was 870), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 247), ProcessCount=168 (was 168), AvailableMemoryMB=1141 (was 1158) 2023-02-13 17:54:42,665 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=395, OpenFileDescriptor=859, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=1141 2023-02-13 17:54:42,665 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:42,669 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:42,669 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:42,670 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:42,672 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:42,672 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:42,674 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:42,674 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b empty. 2023-02-13 17:54:42,675 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:42,675 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 17:54:42,685 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:42,686 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 80781b38cce771d677adbb1da581c54b, NAME => 'testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 80781b38cce771d677adbb1da581c54b, disabling compactions & flushes 2023-02-13 17:54:42,693 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. after waiting 0 ms 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:42,693 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:42,693 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 80781b38cce771d677adbb1da581c54b: Waiting for close lock at 1676310882693Disabling compacts and flushes for region at 1676310882693Disabling writes for close at 1676310882693Writing region close event to WAL at 1676310882693Closed at 1676310882693 2023-02-13 17:54:42,695 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:42,696 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310882695"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310882695"}]},"ts":"1676310882695"} 2023-02-13 17:54:42,697 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:42,698 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:42,698 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310882698"}]},"ts":"1676310882698"} 2023-02-13 17:54:42,699 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 17:54:42,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:42,702 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:42,703 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:42,703 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:42,703 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:42,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, ASSIGN}] 2023-02-13 17:54:42,704 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, ASSIGN 2023-02-13 17:54:42,705 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:42,855 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:42,856 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=80781b38cce771d677adbb1da581c54b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:42,856 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310882856"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310882856"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310882856"}]},"ts":"1676310882856"} 2023-02-13 17:54:42,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 80781b38cce771d677adbb1da581c54b, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:43,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 80781b38cce771d677adbb1da581c54b, NAME => 'testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:43,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:43,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:43,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,017 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,018 DEBUG [StoreOpener-80781b38cce771d677adbb1da581c54b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf1 2023-02-13 17:54:43,018 DEBUG [StoreOpener-80781b38cce771d677adbb1da581c54b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf1 2023-02-13 17:54:43,019 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80781b38cce771d677adbb1da581c54b columnFamilyName cf1 2023-02-13 17:54:43,019 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] regionserver.HStore(310): Store=80781b38cce771d677adbb1da581c54b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,020 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,021 DEBUG [StoreOpener-80781b38cce771d677adbb1da581c54b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf2 2023-02-13 17:54:43,021 DEBUG [StoreOpener-80781b38cce771d677adbb1da581c54b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf2 2023-02-13 17:54:43,021 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 80781b38cce771d677adbb1da581c54b columnFamilyName cf2 2023-02-13 17:54:43,022 INFO [StoreOpener-80781b38cce771d677adbb1da581c54b-1] regionserver.HStore(310): Store=80781b38cce771d677adbb1da581c54b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:43,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:43,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 80781b38cce771d677adbb1da581c54b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60983043, jitterRate=-0.09128184616565704}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:43,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 80781b38cce771d677adbb1da581c54b: Running coprocessor pre-open hook at 1676310883016Writing region info on filesystem at 1676310883016Initializing all the Stores at 1676310883017 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883017Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883017Cleaning up temporary data from old regions at 1676310883024 (+7 ms)Cleaning up detritus from prior splits at 1676310883024Running coprocessor post-open hooks at 1676310883029 (+5 ms)Region opened successfully at 1676310883029 2023-02-13 17:54:43,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b., pid=55, masterSystemTime=1676310883010 2023-02-13 17:54:43,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,032 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=80781b38cce771d677adbb1da581c54b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,032 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310883032"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310883032"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310883032"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310883032"}]},"ts":"1676310883032"} 2023-02-13 17:54:43,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 17:54:43,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 80781b38cce771d677adbb1da581c54b, server=jenkins-hbase4.apache.org,39639,1676310860746 in 175 msec 2023-02-13 17:54:43,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 17:54:43,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, ASSIGN in 332 msec 2023-02-13 17:54:43,037 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:43,037 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310883037"}]},"ts":"1676310883037"} 2023-02-13 17:54:43,039 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 17:54:43,041 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:43,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 372 msec 2023-02-13 17:54:43,171 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,172 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,173 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,174 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,174 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, UNASSIGN}] 2023-02-13 17:54:43,176 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, UNASSIGN 2023-02-13 17:54:43,177 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=80781b38cce771d677adbb1da581c54b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,177 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310883177"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310883177"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310883177"}]},"ts":"1676310883177"} 2023-02-13 17:54:43,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 80781b38cce771d677adbb1da581c54b, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:43,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 80781b38cce771d677adbb1da581c54b, disabling compactions & flushes 2023-02-13 17:54:43,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. after waiting 0 ms 2023-02-13 17:54:43,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 17:54:43,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:43,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b. 2023-02-13 17:54:43,336 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 80781b38cce771d677adbb1da581c54b: Waiting for close lock at 1676310883330Running coprocessor pre-close hooks at 1676310883330Disabling compacts and flushes for region at 1676310883330Disabling writes for close at 1676310883331 (+1 ms)Writing region close event to WAL at 1676310883331Running coprocessor post-close hooks at 1676310883335 (+4 ms)Closed at 1676310883336 (+1 ms) 2023-02-13 17:54:43,337 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:43,338 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=80781b38cce771d677adbb1da581c54b, regionState=CLOSED 2023-02-13 17:54:43,338 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310883338"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310883338"}]},"ts":"1676310883338"} 2023-02-13 17:54:43,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 17:54:43,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 80781b38cce771d677adbb1da581c54b, server=jenkins-hbase4.apache.org,39639,1676310860746 in 161 msec 2023-02-13 17:54:43,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 17:54:43,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=80781b38cce771d677adbb1da581c54b, UNASSIGN in 166 msec 2023-02-13 17:54:43,375 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 17:54:43,377 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 17:54:43,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676310883378"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676310883378"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676310883378"}]},"ts":"1676310883378"} 2023-02-13 17:54:43,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883378"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310883378"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310883378"}]},"ts":"1676310883378"} 2023-02-13 17:54:43,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883378"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310883378"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310883378"}]},"ts":"1676310883378"} 2023-02-13 17:54:43,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, ASSIGN}] 2023-02-13 17:54:43,382 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, ASSIGN 2023-02-13 17:54:43,383 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:43,383 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, ASSIGN 2023-02-13 17:54:43,384 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:43,533 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:54:43,535 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=99f89b3e5ad083d89ee10f03c39f0bb5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,535 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883535"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310883535"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310883535"}]},"ts":"1676310883535"} 2023-02-13 17:54:43,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 99f89b3e5ad083d89ee10f03c39f0bb5, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:43,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=60d3c4b09a775b33cbe1467e336455bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883539"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310883539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310883539"}]},"ts":"1676310883539"} 2023-02-13 17:54:43,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 60d3c4b09a775b33cbe1467e336455bb, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:43,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:43,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 99f89b3e5ad083d89ee10f03c39f0bb5, NAME => 'testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 17:54:43,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:43,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:43,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,694 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,695 DEBUG [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf1 2023-02-13 17:54:43,695 DEBUG [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf1 2023-02-13 17:54:43,696 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99f89b3e5ad083d89ee10f03c39f0bb5 columnFamilyName cf1 2023-02-13 17:54:43,696 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] regionserver.HStore(310): Store=99f89b3e5ad083d89ee10f03c39f0bb5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,696 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,697 DEBUG [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf2 2023-02-13 17:54:43,697 DEBUG [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf2 2023-02-13 17:54:43,698 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99f89b3e5ad083d89ee10f03c39f0bb5 columnFamilyName cf2 2023-02-13 17:54:43,698 INFO [StoreOpener-99f89b3e5ad083d89ee10f03c39f0bb5-1] regionserver.HStore(310): Store=99f89b3e5ad083d89ee10f03c39f0bb5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,699 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:43,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:43,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 99f89b3e5ad083d89ee10f03c39f0bb5; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68787122, jitterRate=0.02500799298286438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:43,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 99f89b3e5ad083d89ee10f03c39f0bb5: Running coprocessor pre-open hook at 1676310883693Writing region info on filesystem at 1676310883693Initializing all the Stores at 1676310883693Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883693Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883694 (+1 ms)Cleaning up temporary data from old regions at 1676310883701 (+7 ms)Cleaning up detritus from prior splits at 1676310883701Running coprocessor post-open hooks at 1676310883704 (+3 ms)Region opened successfully at 1676310883704 2023-02-13 17:54:43,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5., pid=61, masterSystemTime=1676310883688 2023-02-13 17:54:43,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:43,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:43,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:43,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 60d3c4b09a775b33cbe1467e336455bb, NAME => 'testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 17:54:43,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=99f89b3e5ad083d89ee10f03c39f0bb5, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:43,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883706"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310883706"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310883706"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310883706"}]},"ts":"1676310883706"} 2023-02-13 17:54:43,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,708 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,709 DEBUG [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf1 2023-02-13 17:54:43,709 DEBUG [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf1 2023-02-13 17:54:43,710 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d3c4b09a775b33cbe1467e336455bb columnFamilyName cf1 2023-02-13 17:54:43,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-13 17:54:43,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 99f89b3e5ad083d89ee10f03c39f0bb5, server=jenkins-hbase4.apache.org,39639,1676310860746 in 171 msec 2023-02-13 17:54:43,710 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] regionserver.HStore(310): Store=60d3c4b09a775b33cbe1467e336455bb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,710 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, ASSIGN in 329 msec 2023-02-13 17:54:43,712 DEBUG [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf2 2023-02-13 17:54:43,712 DEBUG [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf2 2023-02-13 17:54:43,712 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 60d3c4b09a775b33cbe1467e336455bb columnFamilyName cf2 2023-02-13 17:54:43,713 INFO [StoreOpener-60d3c4b09a775b33cbe1467e336455bb-1] regionserver.HStore(310): Store=60d3c4b09a775b33cbe1467e336455bb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:43,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:43,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:43,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 60d3c4b09a775b33cbe1467e336455bb; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65649708, jitterRate=-0.02174311876296997}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:43,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 60d3c4b09a775b33cbe1467e336455bb: Running coprocessor pre-open hook at 1676310883707Writing region info on filesystem at 1676310883707Initializing all the Stores at 1676310883707Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883708 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310883708Cleaning up temporary data from old regions at 1676310883715 (+7 ms)Cleaning up detritus from prior splits at 1676310883715Running coprocessor post-open hooks at 1676310883718 (+3 ms)Region opened successfully at 1676310883718 2023-02-13 17:54:43,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb., pid=62, masterSystemTime=1676310883688 2023-02-13 17:54:43,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:43,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:43,720 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=60d3c4b09a775b33cbe1467e336455bb, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,721 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883720"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310883720"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310883720"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310883720"}]},"ts":"1676310883720"} 2023-02-13 17:54:43,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-13 17:54:43,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 60d3c4b09a775b33cbe1467e336455bb, server=jenkins-hbase4.apache.org,39639,1676310860746 in 182 msec 2023-02-13 17:54:43,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-13 17:54:43,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, ASSIGN in 343 msec 2023-02-13 17:54:43,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 in 553 msec 2023-02-13 17:54:43,927 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:54:43,930 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:43,931 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 17:54:43,931 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-13 17:54:43,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 17:54:43,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310883935"}]},"ts":"1676310883935"} 2023-02-13 17:54:43,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 17:54:43,938 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 17:54:43,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, UNASSIGN}] 2023-02-13 17:54:43,940 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, UNASSIGN 2023-02-13 17:54:43,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=99f89b3e5ad083d89ee10f03c39f0bb5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310883940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310883940"}]},"ts":"1676310883940"} 2023-02-13 17:54:43,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 99f89b3e5ad083d89ee10f03c39f0bb5, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:43,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, UNASSIGN 2023-02-13 17:54:43,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=60d3c4b09a775b33cbe1467e336455bb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:43,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310883944"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310883944"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310883944"}]},"ts":"1676310883944"} 2023-02-13 17:54:43,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 60d3c4b09a775b33cbe1467e336455bb, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:44,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 17:54:44,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:44,094 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 60d3c4b09a775b33cbe1467e336455bb, disabling compactions & flushes 2023-02-13 17:54:44,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:44,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:44,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. after waiting 0 ms 2023-02-13 17:54:44,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:44,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 17:54:44,099 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:44,100 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb. 2023-02-13 17:54:44,100 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 60d3c4b09a775b33cbe1467e336455bb: Waiting for close lock at 1676310884093Running coprocessor pre-close hooks at 1676310884093Disabling compacts and flushes for region at 1676310884093Disabling writes for close at 1676310884095 (+2 ms)Writing region close event to WAL at 1676310884095Running coprocessor post-close hooks at 1676310884099 (+4 ms)Closed at 1676310884100 (+1 ms) 2023-02-13 17:54:44,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:44,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:44,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 99f89b3e5ad083d89ee10f03c39f0bb5, disabling compactions & flushes 2023-02-13 17:54:44,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:44,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:44,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. after waiting 0 ms 2023-02-13 17:54:44,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:44,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=60d3c4b09a775b33cbe1467e336455bb, regionState=CLOSED 2023-02-13 17:54:44,102 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310884102"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310884102"}]},"ts":"1676310884102"} 2023-02-13 17:54:44,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 17:54:44,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 17:54:44,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 60d3c4b09a775b33cbe1467e336455bb, server=jenkins-hbase4.apache.org,39639,1676310860746 in 159 msec 2023-02-13 17:54:44,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:44,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5. 2023-02-13 17:54:44,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 99f89b3e5ad083d89ee10f03c39f0bb5: Waiting for close lock at 1676310884101Running coprocessor pre-close hooks at 1676310884101Disabling compacts and flushes for region at 1676310884101Disabling writes for close at 1676310884102 (+1 ms)Writing region close event to WAL at 1676310884102Running coprocessor post-close hooks at 1676310884106 (+4 ms)Closed at 1676310884106 2023-02-13 17:54:44,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=60d3c4b09a775b33cbe1467e336455bb, UNASSIGN in 167 msec 2023-02-13 17:54:44,108 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:44,108 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=99f89b3e5ad083d89ee10f03c39f0bb5, regionState=CLOSED 2023-02-13 17:54:44,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310884108"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310884108"}]},"ts":"1676310884108"} 2023-02-13 17:54:44,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 17:54:44,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 99f89b3e5ad083d89ee10f03c39f0bb5, server=jenkins-hbase4.apache.org,39639,1676310860746 in 168 msec 2023-02-13 17:54:44,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-13 17:54:44,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=99f89b3e5ad083d89ee10f03c39f0bb5, UNASSIGN in 171 msec 2023-02-13 17:54:44,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310884112"}]},"ts":"1676310884112"} 2023-02-13 17:54:44,114 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 17:54:44,115 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 17:54:44,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 184 msec 2023-02-13 17:54:44,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 17:54:44,237 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 17:54:44,237 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 17:54:44,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,245 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:44,245 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:44,245 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:44,247 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/recovered.edits] 2023-02-13 17:54:44,248 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/recovered.edits] 2023-02-13 17:54:44,249 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/recovered.edits] 2023-02-13 17:54:44,254 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/recovered.edits/7.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5/recovered.edits/7.seqid 2023-02-13 17:54:44,255 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/recovered.edits/7.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb/recovered.edits/7.seqid 2023-02-13 17:54:44,255 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:54:44,255 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/60d3c4b09a775b33cbe1467e336455bb 2023-02-13 17:54:44,256 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/recovered.edits/4.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b/recovered.edits/4.seqid 2023-02-13 17:54:44,256 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionNoStoreFile/80781b38cce771d677adbb1da581c54b 2023-02-13 17:54:44,256 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 17:54:44,258 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,260 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 17:54:44,262 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 17:54:44,262 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,262 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 17:54:44,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310884262"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:44,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310884262"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:44,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310884262"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:44,264 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:54:44,264 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 80781b38cce771d677adbb1da581c54b, NAME => 'testSplitTableRegionNoStoreFile,,1676310882669.80781b38cce771d677adbb1da581c54b.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 60d3c4b09a775b33cbe1467e336455bb, NAME => 'testSplitTableRegionNoStoreFile,,1676310883172.60d3c4b09a775b33cbe1467e336455bb.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 99f89b3e5ad083d89ee10f03c39f0bb5, NAME => 'testSplitTableRegionNoStoreFile,41,1676310883172.99f89b3e5ad083d89ee10f03c39f0bb5.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 17:54:44,264 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 17:54:44,264 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310884264"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:44,265 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 17:54:44,267 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:54:44,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 30 msec 2023-02-13 17:54:44,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 17:54:44,342 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 17:54:44,356 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400 (was 395) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-852940351_15 at /127.0.0.1:45874 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-852940351_15 at /127.0.0.1:57810 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=846 (was 859), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=1132 (was 1141) 2023-02-13 17:54:44,369 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=400, OpenFileDescriptor=846, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=1131 2023-02-13 17:54:44,370 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:44,373 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:44,374 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:44,376 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,377 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:44,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:44,380 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,380 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece empty. 2023-02-13 17:54:44,381 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 17:54:44,408 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:44,412 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 22d2fa7f26f9d3723b00490163024ece, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 22d2fa7f26f9d3723b00490163024ece, disabling compactions & flushes 2023-02-13 17:54:44,448 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. after waiting 0 ms 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,448 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,448 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 22d2fa7f26f9d3723b00490163024ece: Waiting for close lock at 1676310884448Disabling compacts and flushes for region at 1676310884448Disabling writes for close at 1676310884448Writing region close event to WAL at 1676310884448Closed at 1676310884448 2023-02-13 17:54:44,451 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:44,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310884452"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310884452"}]},"ts":"1676310884452"} 2023-02-13 17:54:44,454 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:44,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:44,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310884455"}]},"ts":"1676310884455"} 2023-02-13 17:54:44,456 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 17:54:44,459 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:44,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:44,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:44,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:44,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:44,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, ASSIGN}] 2023-02-13 17:54:44,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, ASSIGN 2023-02-13 17:54:44,463 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:44,613 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:44,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=22d2fa7f26f9d3723b00490163024ece, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310884614"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310884614"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310884614"}]},"ts":"1676310884614"} 2023-02-13 17:54:44,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 22d2fa7f26f9d3723b00490163024ece, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:44,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 22d2fa7f26f9d3723b00490163024ece, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:44,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:44,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:44,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,772 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,773 DEBUG [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1 2023-02-13 17:54:44,773 DEBUG [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1 2023-02-13 17:54:44,774 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22d2fa7f26f9d3723b00490163024ece columnFamilyName cf1 2023-02-13 17:54:44,774 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] regionserver.HStore(310): Store=22d2fa7f26f9d3723b00490163024ece/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:44,774 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,775 DEBUG [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2 2023-02-13 17:54:44,775 DEBUG [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2 2023-02-13 17:54:44,775 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22d2fa7f26f9d3723b00490163024ece columnFamilyName cf2 2023-02-13 17:54:44,776 INFO [StoreOpener-22d2fa7f26f9d3723b00490163024ece-1] regionserver.HStore(310): Store=22d2fa7f26f9d3723b00490163024ece/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:44,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:44,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:44,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:44,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 22d2fa7f26f9d3723b00490163024ece; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74704452, jitterRate=0.11318308115005493}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:44,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 22d2fa7f26f9d3723b00490163024ece: Running coprocessor pre-open hook at 1676310884771Writing region info on filesystem at 1676310884771Initializing all the Stores at 1676310884772 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310884772Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310884772Cleaning up temporary data from old regions at 1676310884777 (+5 ms)Cleaning up detritus from prior splits at 1676310884778 (+1 ms)Running coprocessor post-open hooks at 1676310884782 (+4 ms)Region opened successfully at 1676310884782 2023-02-13 17:54:44,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece., pid=71, masterSystemTime=1676310884767 2023-02-13 17:54:44,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,785 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=22d2fa7f26f9d3723b00490163024ece, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310884785"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310884785"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310884785"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310884785"}]},"ts":"1676310884785"} 2023-02-13 17:54:44,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 17:54:44,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 22d2fa7f26f9d3723b00490163024ece, server=jenkins-hbase4.apache.org,39639,1676310860746 in 170 msec 2023-02-13 17:54:44,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 17:54:44,790 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, ASSIGN in 327 msec 2023-02-13 17:54:44,790 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:44,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310884790"}]},"ts":"1676310884790"} 2023-02-13 17:54:44,792 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 17:54:44,794 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:44,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 419 msec 2023-02-13 17:54:44,884 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:44,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,885 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:44,885 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:44,885 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:44,885 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:44,886 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,886 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,887 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,887 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,887 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,888 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:44,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:44,889 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:44,890 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,890 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:44,891 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:44,891 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,892 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:44,893 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:54:44,908 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/6b2d7ada625c443d85d9fb62d35995e7 2023-02-13 17:54:44,928 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/7b585d86dd384599a9ec5382355a4546 2023-02-13 17:54:44,934 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/6b2d7ada625c443d85d9fb62d35995e7 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7 2023-02-13 17:54:44,938 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:44,939 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/7b585d86dd384599a9ec5382355a4546 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546 2023-02-13 17:54:44,943 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:54:44,943 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 51ms, sequenceid=6, compaction requested=false 2023-02-13 17:54:44,944 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:44,945 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:44,945 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:44,945 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:44,945 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:44,945 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,945 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:44,945 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:44,947 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,947 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:44,948 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,948 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ea589c2[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:44,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:44,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:44,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:44,951 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,951 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:44,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:44,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:44,953 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:44,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:44,953 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:44,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:44,956 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:44,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:44,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:44,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,956 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:44,959 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:44,960 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,960 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,962 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:44,962 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,962 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:44,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,967 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:44,967 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,967 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:44,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:44,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:44,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:44,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:44,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:44,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:44,968 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,068 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,082 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,084 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,084 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,084 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,084 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:45,084 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,084 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,086 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,086 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,086 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,087 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,087 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,087 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,088 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,088 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:45,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:45,088 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,088 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:45,088 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,092 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,097 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 2023-02-13 17:54:45,110 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/76add9fe356344fdac5230f0be99e394 2023-02-13 17:54:45,114 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 2023-02-13 17:54:45,118 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:45,118 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/76add9fe356344fdac5230f0be99e394 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394 2023-02-13 17:54:45,122 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:54:45,123 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 35ms, sequenceid=19, compaction requested=false 2023-02-13 17:54:45,123 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:45,123 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,125 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:45,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54846384[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:45,127 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:45,127 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:45,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,130 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,131 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,131 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,131 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,131 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,132 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:45,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:45,135 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:45,135 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:45,135 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:45,135 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,135 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,137 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,137 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,139 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,139 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,142 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:45,142 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:45,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,243 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,256 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,258 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:45,259 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,259 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,260 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,260 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,260 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,262 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,262 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,262 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,263 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:45,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:45,265 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,265 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:45,265 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:45,265 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,266 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,266 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,266 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,267 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,267 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,280 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/036adfc806f34c21acf58f1c0e84e4e4 2023-02-13 17:54:45,294 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 2023-02-13 17:54:45,299 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/036adfc806f34c21acf58f1c0e84e4e4 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4 2023-02-13 17:54:45,303 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:45,304 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 2023-02-13 17:54:45,308 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:54:45,311 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 46ms, sequenceid=32, compaction requested=true 2023-02-13 17:54:45,311 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:45,311 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,311 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,316 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,316 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,316 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,316 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:45,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@777c8ee4[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:45,318 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:45,318 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,320 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,320 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:45,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,320 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,321 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,321 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:45,322 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,322 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,322 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,324 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,324 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,324 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,324 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,325 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,325 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:45,327 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:45,327 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:45,327 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:45,328 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:45,328 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,328 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,329 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,330 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,330 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,331 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,332 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,333 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,333 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,335 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,335 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,335 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,335 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,335 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,336 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,336 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:45,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:45,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,336 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,337 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,437 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,450 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,451 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:45,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,454 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,454 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,455 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,456 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,456 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,456 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:45,457 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:45,457 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,457 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:45,457 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,459 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,459 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,459 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,459 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,467 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/655e2da01c194b66b2f3ff791d4864d8 2023-02-13 17:54:45,479 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca 2023-02-13 17:54:45,484 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/655e2da01c194b66b2f3ff791d4864d8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8 2023-02-13 17:54:45,488 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:45,489 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca 2023-02-13 17:54:45,493 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:54:45,494 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 37ms, sequenceid=45, compaction requested=true 2023-02-13 17:54:45,494 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:45,494 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,495 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:45,495 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:45,495 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,495 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,495 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,496 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,496 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,496 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,496 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:45,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@494246a4[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:45,498 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:45,498 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,499 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,499 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,499 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:45,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,500 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,500 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:45,500 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,502 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,503 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,503 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,503 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,503 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,503 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,503 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,504 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:45,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:45,506 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:45,506 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:45,506 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:45,506 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,506 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,513 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,515 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,515 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,515 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,517 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,518 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,518 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:45,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:45,518 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,519 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,519 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:45,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,519 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,519 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,519 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,619 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,632 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,633 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:45,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,635 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,635 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,635 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,636 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,637 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,637 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,638 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,638 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:45,638 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:45,638 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,638 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:45,638 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:45,640 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,642 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,647 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/0dbc2409ee1b44f590fcffcf147bffe8 2023-02-13 17:54:45,662 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/fe38f26fae7941b69b9c00e473bed2c4 2023-02-13 17:54:45,666 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/0dbc2409ee1b44f590fcffcf147bffe8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8 2023-02-13 17:54:45,670 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:45,671 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/fe38f26fae7941b69b9c00e473bed2c4 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4 2023-02-13 17:54:45,675 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:54:45,676 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 38ms, sequenceid=58, compaction requested=true 2023-02-13 17:54:45,676 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:45,676 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,676 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:45,676 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:45,676 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,676 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,676 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,677 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,677 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,678 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:45,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bfb3dd7[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:45,680 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:45,680 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,681 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,681 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:45,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,681 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,682 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:45,682 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,683 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,683 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,689 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,689 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,689 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:45,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:45,692 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:45,692 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:45,693 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:45,693 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,693 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,694 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,695 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,696 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,696 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,698 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,698 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,698 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,702 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,701 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:45,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:45,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,702 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:45,803 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,816 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,818 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,818 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,818 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,818 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:45,818 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,819 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,828 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,828 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,828 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,828 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,830 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:45,830 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,830 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:45,830 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,830 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,830 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:45,830 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,832 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,839 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc 2023-02-13 17:54:45,851 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/00844ebbde6b4b06815bf7f66b8d3922 2023-02-13 17:54:45,855 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc 2023-02-13 17:54:45,859 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:45,859 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/00844ebbde6b4b06815bf7f66b8d3922 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922 2023-02-13 17:54:45,863 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:54:45,864 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 34ms, sequenceid=71, compaction requested=true 2023-02-13 17:54:45,864 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:45,864 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:45,864 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:45,866 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,866 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:45,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c14e56f[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:45,867 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:45,868 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:45,869 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,869 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:45,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,870 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:45,869 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:45,870 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:45,870 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,872 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,873 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:45,876 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:45,876 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:45,876 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:45,876 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,876 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,878 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,879 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:45,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,879 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,881 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,881 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:45,881 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,884 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:45,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,884 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:45,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,884 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,885 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:45,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:45,985 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:45,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:45,998 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:45,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:45,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:45,999 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:45,999 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:45,999 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:45,999 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:46,000 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,000 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,002 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,002 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,002 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,003 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:46,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,004 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,004 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,004 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:46,005 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,006 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,014 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,018 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,025 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,029 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,030 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/0a415759c9684a39a2db5eb2667648b2 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,033 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,033 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 17:54:46,034 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/14efa4fa35df4c1e9d07a357c465cc20 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,037 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,037 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 17:54:46,038 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 34ms, sequenceid=83, compaction requested=true 2023-02-13 17:54:46,038 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:46,038 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,038 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:46,038 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:46,038 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,038 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,038 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,040 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,040 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,040 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,040 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:46,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f0c4da7[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:46,042 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:46,042 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,043 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,043 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:46,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,043 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,043 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:46,043 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,044 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,044 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:46,043 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,044 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,044 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:46,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,046 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,050 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:46,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:46,053 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:46,053 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:46,053 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:46,053 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,053 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:46,054 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,054 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,055 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,054 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,054 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,055 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,055 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,055 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,056 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,057 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,057 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,057 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,061 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,061 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:46,061 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,062 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,062 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:46,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,061 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,062 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,062 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,062 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,162 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:46,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:46,170 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:46,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:46,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,171 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:46,171 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,171 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:46,171 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:46,172 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,172 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,173 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,173 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,173 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,174 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:46,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:46,175 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:46,175 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,175 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:46,176 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 17:54:46,176 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,176 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,177 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,184 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,188 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,195 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,202 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,202 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/8c75eaa78df44a22af0d9441d98c0db8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,206 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,206 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 17:54:46,206 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/97f857b13e56474aa93439f95348c364 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,211 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,211 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 17:54:46,211 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 36ms, sequenceid=91, compaction requested=true 2023-02-13 17:54:46,211 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:46,212 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,212 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,213 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,213 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,213 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,213 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:46,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24205384[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:46,215 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:46,215 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:46,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,217 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,217 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:46,217 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,218 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:46,218 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,218 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,219 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,219 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,221 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:46,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:46,224 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:46,224 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:46,224 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:46,224 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,224 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,226 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,226 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,226 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,228 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,228 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,229 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,234 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,234 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,235 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,235 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:46,235 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:46,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,235 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:46,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,236 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:46,336 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:46,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:46,345 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:54:46,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:54:46,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,346 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:46,346 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,346 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 17:54:46,346 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 17:54:46,347 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,347 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,348 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,348 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,348 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:54:46,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,349 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,350 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,350 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,350 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:54:46,350 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. started... 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,350 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,350 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 17:54:46,352 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,352 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,352 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,352 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,353 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,353 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,359 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,363 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,372 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,375 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,376 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/9038a883dd904e3d975176310bb4c01f as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,380 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,380 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 17:54:46,381 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/6aaaca660ed34746940ed504690bd286 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,385 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,385 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 17:54:46,386 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 36ms, sequenceid=99, compaction requested=true 2023-02-13 17:54:46,386 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 22d2fa7f26f9d3723b00490163024ece: 2023-02-13 17:54:46,386 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,386 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:54:46,386 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:54:46,386 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,386 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 17:54:46,386 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 17:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,389 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 17:54:46,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66f7c752[Count = 0] remaining members to acquire global barrier 2023-02-13 17:54:46,391 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 17:54:46,391 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,393 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:54:46,392 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:54:46,392 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 17:54:46,393 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:54:46,394 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,394 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,394 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,395 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 17:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:54:46,398 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 17:54:46,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:54:46,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:54:46,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,398 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,400 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,400 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:54:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:54:46,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,426 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:54:46,426 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:54:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,428 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:54:46,428 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,429 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,432 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 17:54:46,432 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:54:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,432 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:54:46,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:46,533 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:54:46,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:54:46,536 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,537 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:46,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:46,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:46,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:46,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, UNASSIGN}] 2023-02-13 17:54:46,540 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, UNASSIGN 2023-02-13 17:54:46,540 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=22d2fa7f26f9d3723b00490163024ece, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:46,541 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310886540"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310886540"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310886540"}]},"ts":"1676310886540"} 2023-02-13 17:54:46,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 22d2fa7f26f9d3723b00490163024ece, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:46,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 22d2fa7f26f9d3723b00490163024ece, disabling compactions & flushes 2023-02-13 17:54:46,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. after waiting 0 ms 2023-02-13 17:54:46,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 22d2fa7f26f9d3723b00490163024ece 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 17:54:46,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf1/e0dc27e44c6642cb94aea8bc4f050981 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 17:54:46,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.tmp/cf2/ee3b7b2fb97d48f5b012ef975d842862 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 17:54:46,728 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 34ms, sequenceid=103, compaction requested=true 2023-02-13 17:54:46,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 17:54:46,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:46,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. 2023-02-13 17:54:46,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 22d2fa7f26f9d3723b00490163024ece: Waiting for close lock at 1676310886693Running coprocessor pre-close hooks at 1676310886693Disabling compacts and flushes for region at 1676310886693Disabling writes for close at 1676310886694 (+1 ms)Obtaining lock to block concurrent updates at 1676310886694Preparing flush snapshotting stores in 22d2fa7f26f9d3723b00490163024ece at 1676310886694Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676310886694Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece. at 1676310886694Flushing 22d2fa7f26f9d3723b00490163024ece/cf1: creating writer at 1676310886696 (+2 ms)Flushing 22d2fa7f26f9d3723b00490163024ece/cf1: appending metadata at 1676310886697 (+1 ms)Flushing 22d2fa7f26f9d3723b00490163024ece/cf1: closing flushed file at 1676310886697Flushing 22d2fa7f26f9d3723b00490163024ece/cf2: creating writer at 1676310886709 (+12 ms)Flushing 22d2fa7f26f9d3723b00490163024ece/cf2: appending metadata at 1676310886711 (+2 ms)Flushing 22d2fa7f26f9d3723b00490163024ece/cf2: closing flushed file at 1676310886711Flushing 22d2fa7f26f9d3723b00490163024ece/cf1: reopening flushed file at 1676310886721 (+10 ms)Flushing 22d2fa7f26f9d3723b00490163024ece/cf2: reopening flushed file at 1676310886725 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 22d2fa7f26f9d3723b00490163024ece in 34ms, sequenceid=103, compaction requested=true at 1676310886728 (+3 ms)Writing region close event to WAL at 1676310886731 (+3 ms)Running coprocessor post-close hooks at 1676310886734 (+3 ms)Closed at 1676310886734 2023-02-13 17:54:46,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,735 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=22d2fa7f26f9d3723b00490163024ece, regionState=CLOSED 2023-02-13 17:54:46,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310886735"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310886735"}]},"ts":"1676310886735"} 2023-02-13 17:54:46,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 17:54:46,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 22d2fa7f26f9d3723b00490163024ece, server=jenkins-hbase4.apache.org,39639,1676310860746 in 194 msec 2023-02-13 17:54:46,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 17:54:46,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=22d2fa7f26f9d3723b00490163024ece, UNASSIGN in 199 msec 2023-02-13 17:54:46,753 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=22d2fa7f26f9d3723b00490163024ece, threads=16 2023-02-13 17:54:46,753 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,753 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,753 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,753 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,753 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,754 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,756 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,756 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,760 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,760 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,760 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,760 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,764 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,764 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,768 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,768 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,772 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,773 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,775 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca, top=false 2023-02-13 17:54:46,776 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8, top=false 2023-02-13 17:54:46,776 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922, top=false 2023-02-13 17:54:46,781 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,783 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,785 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97f857b13e56474aa93439f95348c364 2023-02-13 17:54:46,785 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364, top=true 2023-02-13 17:54:46,785 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4, top=false 2023-02-13 17:54:46,786 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:46,786 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286, top=true 2023-02-13 17:54:46,793 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,796 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8, top=false 2023-02-13 17:54:46,796 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc, top=false 2023-02-13 17:54:46,810 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:46,810 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862, top=true 2023-02-13 17:54:46,831 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,835 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,835 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,838 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,841 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:46,841 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8, top=true 2023-02-13 17:54:46,846 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8 for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,850 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,850 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,853 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,856 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:46,856 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f, top=true 2023-02-13 17:54:46,870 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8 for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,870 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364 for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,870 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,870 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,873 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,873 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,876 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862 for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,876 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,876 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,880 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:46,880 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981, top=true 2023-02-13 17:54:46,888 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4 for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,896 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922 for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,900 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8 for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,900 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286 for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,900 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,900 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,904 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc for child: db35248b07d43ec3d6842233230b451a, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,914 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,914 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:46,914 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,914 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:46,916 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,932 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,932 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,940 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/.splits/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981 for child: 5bd57f0845d56cf0851dc8b161781e6d, parent: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,940 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,956 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,958 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,958 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,972 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,972 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,978 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,984 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,986 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 for region: 22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:46,986 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 22d2fa7f26f9d3723b00490163024ece Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:54:47,023 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 17:54:47,025 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 17:54:47,026 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676310887026"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676310887026"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676310887026"}]},"ts":"1676310887026"} 2023-02-13 17:54:47,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887026"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310887026"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310887026"}]},"ts":"1676310887026"} 2023-02-13 17:54:47,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887026"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310887026"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310887026"}]},"ts":"1676310887026"} 2023-02-13 17:54:47,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, ASSIGN}] 2023-02-13 17:54:47,031 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, ASSIGN 2023-02-13 17:54:47,031 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:47,032 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, ASSIGN 2023-02-13 17:54:47,033 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:54:47,181 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:54:47,183 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=db35248b07d43ec3d6842233230b451a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,183 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887183"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310887183"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310887183"}]},"ts":"1676310887183"} 2023-02-13 17:54:47,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure db35248b07d43ec3d6842233230b451a, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:47,187 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=5bd57f0845d56cf0851dc8b161781e6d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,187 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887187"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310887187"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310887187"}]},"ts":"1676310887187"} 2023-02-13 17:54:47,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 5bd57f0845d56cf0851dc8b161781e6d, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:47,280 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:54:47,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => db35248b07d43ec3d6842233230b451a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 17:54:47,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:47,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:47,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,341 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,341 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1 2023-02-13 17:54:47,342 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1 2023-02-13 17:54:47,342 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db35248b07d43ec3d6842233230b451a columnFamilyName cf1 2023-02-13 17:54:47,351 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-bottom 2023-02-13 17:54:47,354 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,355 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-bottom 2023-02-13 17:54:47,358 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-bottom 2023-02-13 17:54:47,362 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-bottom 2023-02-13 17:54:47,365 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8 2023-02-13 17:54:47,369 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc 2023-02-13 17:54:47,372 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8 2023-02-13 17:54:47,372 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(310): Store=db35248b07d43ec3d6842233230b451a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:47,372 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,373 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2 2023-02-13 17:54:47,373 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2 2023-02-13 17:54:47,373 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region db35248b07d43ec3d6842233230b451a columnFamilyName cf2 2023-02-13 17:54:47,384 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,385 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-bottom 2023-02-13 17:54:47,390 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-bottom 2023-02-13 17:54:47,395 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-bottom 2023-02-13 17:54:47,400 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-bottom 2023-02-13 17:54:47,403 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922 2023-02-13 17:54:47,407 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca 2023-02-13 17:54:47,409 DEBUG [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4 2023-02-13 17:54:47,410 INFO [StoreOpener-db35248b07d43ec3d6842233230b451a-1] regionserver.HStore(310): Store=db35248b07d43ec3d6842233230b451a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:47,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:47,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened db35248b07d43ec3d6842233230b451a; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69368392, jitterRate=0.03366959095001221}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:47,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for db35248b07d43ec3d6842233230b451a: Running coprocessor pre-open hook at 1676310887340Writing region info on filesystem at 1676310887340Initializing all the Stores at 1676310887340Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310887340Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310887341 (+1 ms)Cleaning up temporary data from old regions at 1676310887412 (+71 ms)Cleaning up detritus from prior splits at 1676310887412Running coprocessor post-open hooks at 1676310887414 (+2 ms)Region opened successfully at 1676310887414 2023-02-13 17:54:47,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a., pid=77, masterSystemTime=1676310887336 2023-02-13 17:54:47,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:47,416 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:47,416 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,416 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): db35248b07d43ec3d6842233230b451a/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of db35248b07d43ec3d6842233230b451a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,416 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): db35248b07d43ec3d6842233230b451a/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of db35248b07d43ec3d6842233230b451a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp, totalSize=35.0 K 2023-02-13 17:54:47,416 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp, totalSize=35.0 K 2023-02-13 17:54:47,417 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310884878 2023-02-13 17:54:47,417 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310884878 2023-02-13 17:54:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,417 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310885070 2023-02-13 17:54:47,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5bd57f0845d56cf0851dc8b161781e6d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 17:54:47,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=db35248b07d43ec3d6842233230b451a, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,417 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310885070 2023-02-13 17:54:47,418 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887417"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310887417"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310887417"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310887417"}]},"ts":"1676310887417"} 2023-02-13 17:54:47,418 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310885245 2023-02-13 17:54:47,418 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310885245 2023-02-13 17:54:47,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:47,418 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310885438 2023-02-13 17:54:47,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,418 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310885438 2023-02-13 17:54:47,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:47,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,419 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310885620 2023-02-13 17:54:47,419 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310885620 2023-02-13 17:54:47,419 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310885805 2023-02-13 17:54:47,419 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310885805 2023-02-13 17:54:47,420 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310885986 2023-02-13 17:54:47,420 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,420 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310885986 2023-02-13 17:54:47,422 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1 2023-02-13 17:54:47,422 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1 2023-02-13 17:54:47,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-13 17:54:47,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure db35248b07d43ec3d6842233230b451a, server=jenkins-hbase4.apache.org,39639,1676310860746 in 234 msec 2023-02-13 17:54:47,423 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5bd57f0845d56cf0851dc8b161781e6d columnFamilyName cf1 2023-02-13 17:54:47,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, ASSIGN in 393 msec 2023-02-13 17:54:47,436 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): db35248b07d43ec3d6842233230b451a#cf2#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,436 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): db35248b07d43ec3d6842233230b451a#cf1#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,437 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-top 2023-02-13 17:54:47,440 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,440 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-top 2023-02-13 17:54:47,443 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-top 2023-02-13 17:54:47,446 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-top 2023-02-13 17:54:47,449 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:47,449 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:47,454 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:47,454 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:47,457 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:47,457 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:47,460 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(310): Store=5bd57f0845d56cf0851dc8b161781e6d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:47,460 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,461 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2 2023-02-13 17:54:47,461 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2 2023-02-13 17:54:47,461 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5bd57f0845d56cf0851dc8b161781e6d columnFamilyName cf2 2023-02-13 17:54:47,465 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp/cf2/bcd1d700dd0b46be83fe956200055b67 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/bcd1d700dd0b46be83fe956200055b67 2023-02-13 17:54:47,469 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp/cf1/c269e0b8e2524b6cbec42afbd8a4f974 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/c269e0b8e2524b6cbec42afbd8a4f974 2023-02-13 17:54:47,472 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db35248b07d43ec3d6842233230b451a/cf2 of db35248b07d43ec3d6842233230b451a into bcd1d700dd0b46be83fe956200055b67(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,472 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for db35248b07d43ec3d6842233230b451a: 2023-02-13 17:54:47,472 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a., storeName=db35248b07d43ec3d6842233230b451a/cf2, priority=9, startTime=1676310887416; duration=0sec 2023-02-13 17:54:47,472 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,473 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,473 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-top 2023-02-13 17:54:47,474 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in db35248b07d43ec3d6842233230b451a/cf1 of db35248b07d43ec3d6842233230b451a into c269e0b8e2524b6cbec42afbd8a4f974(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,474 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for db35248b07d43ec3d6842233230b451a: 2023-02-13 17:54:47,474 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a., storeName=db35248b07d43ec3d6842233230b451a/cf1, priority=9, startTime=1676310887415; duration=0sec 2023-02-13 17:54:47,474 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,476 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-top 2023-02-13 17:54:47,479 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-top 2023-02-13 17:54:47,482 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-top 2023-02-13 17:54:47,487 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:47,487 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:47,490 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97f857b13e56474aa93439f95348c364 2023-02-13 17:54:47,490 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364 2023-02-13 17:54:47,493 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:47,493 DEBUG [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:47,493 INFO [StoreOpener-5bd57f0845d56cf0851dc8b161781e6d-1] regionserver.HStore(310): Store=5bd57f0845d56cf0851dc8b161781e6d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:47,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:54:47,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5bd57f0845d56cf0851dc8b161781e6d; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66902759, jitterRate=-0.003071203827857971}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:54:47,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5bd57f0845d56cf0851dc8b161781e6d: Running coprocessor pre-open hook at 1676310887419Writing region info on filesystem at 1676310887419Initializing all the Stores at 1676310887419Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310887419Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310887420 (+1 ms)Cleaning up temporary data from old regions at 1676310887495 (+75 ms)Cleaning up detritus from prior splits at 1676310887495Running coprocessor post-open hooks at 1676310887497 (+2 ms)Region opened successfully at 1676310887497 2023-02-13 17:54:47,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d., pid=78, masterSystemTime=1676310887336 2023-02-13 17:54:47,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:47,499 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:47,499 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,499 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): 5bd57f0845d56cf0851dc8b161781e6d/cf1 is initiating minor compaction (all files) 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5bd57f0845d56cf0851dc8b161781e6d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,499 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): 5bd57f0845d56cf0851dc8b161781e6d/cf2 is initiating minor compaction (all files) 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5bd57f0845d56cf0851dc8b161781e6d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp, totalSize=34.8 K 2023-02-13 17:54:47,499 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp, totalSize=34.8 K 2023-02-13 17:54:47,499 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310884878 2023-02-13 17:54:47,500 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310884878 2023-02-13 17:54:47,500 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310885070 2023-02-13 17:54:47,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,500 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310885070 2023-02-13 17:54:47,501 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310885245 2023-02-13 17:54:47,501 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=5bd57f0845d56cf0851dc8b161781e6d, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,501 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310885245 2023-02-13 17:54:47,501 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887501"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310887501"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310887501"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310887501"}]},"ts":"1676310887501"} 2023-02-13 17:54:47,501 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676310885986 2023-02-13 17:54:47,501 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,501 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676310885986 2023-02-13 17:54:47,502 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,502 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,502 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,502 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,503 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-13 17:54:47,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 5bd57f0845d56cf0851dc8b161781e6d, server=jenkins-hbase4.apache.org,39639,1676310860746 in 314 msec 2023-02-13 17:54:47,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-02-13 17:54:47,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, ASSIGN in 475 msec 2023-02-13 17:54:47,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d in 971 msec 2023-02-13 17:54:47,524 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): 5bd57f0845d56cf0851dc8b161781e6d#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,524 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5bd57f0845d56cf0851dc8b161781e6d#cf1#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,546 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c516cd5a412947db89bc4f8cc088967d: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,547 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp/cf1/c516cd5a412947db89bc4f8cc088967d as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/c516cd5a412947db89bc4f8cc088967d 2023-02-13 17:54:47,547 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 1b04e9fc32104ae383534b73a6386530: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,548 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp/cf2/1b04e9fc32104ae383534b73a6386530 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/1b04e9fc32104ae383534b73a6386530 2023-02-13 17:54:47,550 DEBUG [Listener at localhost/44157] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 17:54:47,552 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c516cd5a412947db89bc4f8cc088967d: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,554 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5bd57f0845d56cf0851dc8b161781e6d/cf1 of 5bd57f0845d56cf0851dc8b161781e6d into c516cd5a412947db89bc4f8cc088967d(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,554 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5bd57f0845d56cf0851dc8b161781e6d: 2023-02-13 17:54:47,554 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52156, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 17:54:47,554 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d., storeName=5bd57f0845d56cf0851dc8b161781e6d/cf1, priority=9, startTime=1676310887498; duration=0sec 2023-02-13 17:54:47,554 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 1b04e9fc32104ae383534b73a6386530: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,554 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,555 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5bd57f0845d56cf0851dc8b161781e6d/cf2 of 5bd57f0845d56cf0851dc8b161781e6d into 1b04e9fc32104ae383534b73a6386530(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,555 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5bd57f0845d56cf0851dc8b161781e6d: 2023-02-13 17:54:47,555 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d., storeName=5bd57f0845d56cf0851dc8b161781e6d/cf2, priority=9, startTime=1676310887499; duration=0sec 2023-02-13 17:54:47,555 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,556 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,557 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 17:54:47,557 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.HStore(1912): db35248b07d43ec3d6842233230b451a/cf1 is initiating major compaction (all files) 2023-02-13 17:54:47,557 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@73273e59; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,557 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of db35248b07d43ec3d6842233230b451a/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,557 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/c269e0b8e2524b6cbec42afbd8a4f974] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp, totalSize=6.8 K 2023-02-13 17:54:47,557 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 17:54:47,558 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.HStore(1912): db35248b07d43ec3d6842233230b451a/cf2 is initiating major compaction (all files) 2023-02-13 17:54:47,558 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@317aaeec; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,558 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of db35248b07d43ec3d6842233230b451a/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,558 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/bcd1d700dd0b46be83fe956200055b67] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp, totalSize=6.8 K 2023-02-13 17:54:47,558 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting c269e0b8e2524b6cbec42afbd8a4f974, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310884878 2023-02-13 17:54:47,558 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting bcd1d700dd0b46be83fe956200055b67, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310884878 2023-02-13 17:54:47,560 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.HStore(1912): 5bd57f0845d56cf0851dc8b161781e6d/cf1 is initiating major compaction (all files) 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@74d7b7bb; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.HStore(1912): 5bd57f0845d56cf0851dc8b161781e6d/cf2 is initiating major compaction (all files) 2023-02-13 17:54:47,560 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@641787e9; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 17:54:47,561 INFO [Listener at localhost/44157] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 17:54:47,565 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): db35248b07d43ec3d6842233230b451a#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,568 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): db35248b07d43ec3d6842233230b451a#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. is not mergeable because it has references 2023-02-13 17:54:47,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. is not mergeable because it has references 2023-02-13 17:54:47,589 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp/cf1/180726ec8b01488ca47acf8473048747 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/180726ec8b01488ca47acf8473048747 2023-02-13 17:54:47,589 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/.tmp/cf2/1fc1628f41784f59ad07457a99b40da8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/1fc1628f41784f59ad07457a99b40da8 2023-02-13 17:54:47,594 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in db35248b07d43ec3d6842233230b451a/cf2 of db35248b07d43ec3d6842233230b451a into 1fc1628f41784f59ad07457a99b40da8(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,594 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in db35248b07d43ec3d6842233230b451a/cf1 of db35248b07d43ec3d6842233230b451a into 180726ec8b01488ca47acf8473048747(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,594 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for db35248b07d43ec3d6842233230b451a: 2023-02-13 17:54:47,594 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for db35248b07d43ec3d6842233230b451a: 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676310887557; duration=0sec 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676310887557; duration=0sec 2023-02-13 17:54:47,596 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 17:54:47,596 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5bd57f0845d56cf0851dc8b161781e6d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5bd57f0845d56cf0851dc8b161781e6d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/1b04e9fc32104ae383534b73a6386530] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp, totalSize=5.0 K 2023-02-13 17:54:47,596 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/c516cd5a412947db89bc4f8cc088967d] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp, totalSize=5.0 K 2023-02-13 17:54:47,596 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 1b04e9fc32104ae383534b73a6386530, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,597 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting c516cd5a412947db89bc4f8cc088967d, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 17:54:47,599 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5bd57f0845d56cf0851dc8b161781e6d#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,599 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): 5bd57f0845d56cf0851dc8b161781e6d#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:54:47,609 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 70ccb7f180284f229f7adc5d5271bb44: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,609 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 61727d04f0164bd09c492ec63cf7eb9c: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,609 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp/cf2/70ccb7f180284f229f7adc5d5271bb44 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/70ccb7f180284f229f7adc5d5271bb44 2023-02-13 17:54:47,610 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/.tmp/cf1/61727d04f0164bd09c492ec63cf7eb9c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/61727d04f0164bd09c492ec63cf7eb9c 2023-02-13 17:54:47,613 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 70ccb7f180284f229f7adc5d5271bb44: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,613 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 61727d04f0164bd09c492ec63cf7eb9c: NONE, but ROW specified in column family configuration 2023-02-13 17:54:47,614 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 5bd57f0845d56cf0851dc8b161781e6d/cf2 of 5bd57f0845d56cf0851dc8b161781e6d into 70ccb7f180284f229f7adc5d5271bb44(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,614 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 5bd57f0845d56cf0851dc8b161781e6d/cf1 of 5bd57f0845d56cf0851dc8b161781e6d into 61727d04f0164bd09c492ec63cf7eb9c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:54:47,614 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5bd57f0845d56cf0851dc8b161781e6d: 2023-02-13 17:54:47,614 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5bd57f0845d56cf0851dc8b161781e6d: 2023-02-13 17:54:47,614 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676310887560; duration=0sec 2023-02-13 17:54:47,614 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676310887560; duration=0sec 2023-02-13 17:54:47,615 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,615 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:54:47,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39639] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. is not mergeable because it has references 2023-02-13 17:54:47,692 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39639] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. is not mergeable because it has references 2023-02-13 17:54:47,699 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 17:54:47,700 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:47,702 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:47,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:47,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:47,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 17:54:47,705 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310887705"}]},"ts":"1676310887705"} 2023-02-13 17:54:47,706 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 17:54:47,708 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 17:54:47,709 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, UNASSIGN}] 2023-02-13 17:54:47,709 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, UNASSIGN 2023-02-13 17:54:47,710 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=5bd57f0845d56cf0851dc8b161781e6d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887710"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310887710"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310887710"}]},"ts":"1676310887710"} 2023-02-13 17:54:47,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 5bd57f0845d56cf0851dc8b161781e6d, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:47,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, UNASSIGN 2023-02-13 17:54:47,713 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=db35248b07d43ec3d6842233230b451a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:47,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887713"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310887713"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310887713"}]},"ts":"1676310887713"} 2023-02-13 17:54:47,715 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure db35248b07d43ec3d6842233230b451a, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:47,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 17:54:47,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5bd57f0845d56cf0851dc8b161781e6d, disabling compactions & flushes 2023-02-13 17:54:47,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. after waiting 0 ms 2023-02-13 17:54:47,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,863 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/c516cd5a412947db89bc4f8cc088967d, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981] to archive 2023-02-13 17:54:47,864 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:47,866 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,867 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,868 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,869 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,870 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:47,871 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:47,872 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/c516cd5a412947db89bc4f8cc088967d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/c516cd5a412947db89bc4f8cc088967d 2023-02-13 17:54:47,873 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:47,874 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/1b04e9fc32104ae383534b73a6386530, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862] to archive 2023-02-13 17:54:47,875 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:47,876 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,877 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,878 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,879 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,880 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-97f857b13e56474aa93439f95348c364 2023-02-13 17:54:47,881 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:47,882 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/1b04e9fc32104ae383534b73a6386530 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/1b04e9fc32104ae383534b73a6386530 2023-02-13 17:54:47,882 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:47,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 17:54:47,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:47,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d. 2023-02-13 17:54:47,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5bd57f0845d56cf0851dc8b161781e6d: Waiting for close lock at 1676310887863Running coprocessor pre-close hooks at 1676310887863Disabling compacts and flushes for region at 1676310887863Disabling writes for close at 1676310887863Writing region close event to WAL at 1676310887883 (+20 ms)Running coprocessor post-close hooks at 1676310887886 (+3 ms)Closed at 1676310887886 2023-02-13 17:54:47,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:47,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing db35248b07d43ec3d6842233230b451a, disabling compactions & flushes 2023-02-13 17:54:47,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. after waiting 0 ms 2023-02-13 17:54:47,888 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=5bd57f0845d56cf0851dc8b161781e6d, regionState=CLOSED 2023-02-13 17:54:47,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887888"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310887888"}]},"ts":"1676310887888"} 2023-02-13 17:54:47,889 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/c269e0b8e2524b6cbec42afbd8a4f974, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2-bottom] to archive 2023-02-13 17:54:47,889 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:47,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 17:54:47,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 5bd57f0845d56cf0851dc8b161781e6d, server=jenkins-hbase4.apache.org,39639,1676310860746 in 178 msec 2023-02-13 17:54:47,891 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/6b2d7ada625c443d85d9fb62d35995e7.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=5bd57f0845d56cf0851dc8b161781e6d, UNASSIGN in 182 msec 2023-02-13 17:54:47,892 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/a52ff3b1f14041d6b31e6402d4a25cd3.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,893 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/036adfc806f34c21acf58f1c0e84e4e4.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,894 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-655e2da01c194b66b2f3ff791d4864d8 2023-02-13 17:54:47,895 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-0dbc2409ee1b44f590fcffcf147bffe8 2023-02-13 17:54:47,895 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-332b1dcc2c074a2c8ce8a69f627fb7fc 2023-02-13 17:54:47,896 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/c269e0b8e2524b6cbec42afbd8a4f974 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/c269e0b8e2524b6cbec42afbd8a4f974 2023-02-13 17:54:47,897 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/0a415759c9684a39a2db5eb2667648b2.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,899 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/bcd1d700dd0b46be83fe956200055b67, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20-bottom] to archive 2023-02-13 17:54:47,899 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:54:47,900 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/7b585d86dd384599a9ec5382355a4546.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,901 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/76add9fe356344fdac5230f0be99e394.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,902 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/cd20a0c73e2e402d89d066b10dbc1ca1.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,903 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-5ffa2c5e7ac242b0b97ea8dea04445ca 2023-02-13 17:54:47,904 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-fe38f26fae7941b69b9c00e473bed2c4 2023-02-13 17:54:47,905 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/testSplitTableRegionDeletedRowsDaughter=22d2fa7f26f9d3723b00490163024ece-00844ebbde6b4b06815bf7f66b8d3922 2023-02-13 17:54:47,906 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/bcd1d700dd0b46be83fe956200055b67 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/bcd1d700dd0b46be83fe956200055b67 2023-02-13 17:54:47,906 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/14efa4fa35df4c1e9d07a357c465cc20.22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:47,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 17:54:47,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:47,910 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a. 2023-02-13 17:54:47,910 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for db35248b07d43ec3d6842233230b451a: Waiting for close lock at 1676310887888Running coprocessor pre-close hooks at 1676310887888Disabling compacts and flushes for region at 1676310887888Disabling writes for close at 1676310887888Writing region close event to WAL at 1676310887907 (+19 ms)Running coprocessor post-close hooks at 1676310887910 (+3 ms)Closed at 1676310887910 2023-02-13 17:54:47,911 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:47,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=db35248b07d43ec3d6842233230b451a, regionState=CLOSED 2023-02-13 17:54:47,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676310887911"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310887911"}]},"ts":"1676310887911"} 2023-02-13 17:54:47,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 17:54:47,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure db35248b07d43ec3d6842233230b451a, server=jenkins-hbase4.apache.org,39639,1676310860746 in 199 msec 2023-02-13 17:54:47,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 17:54:47,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=db35248b07d43ec3d6842233230b451a, UNASSIGN in 205 msec 2023-02-13 17:54:47,916 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310887916"}]},"ts":"1676310887916"} 2023-02-13 17:54:47,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 17:54:47,918 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 17:54:47,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 216 msec 2023-02-13 17:54:48,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 17:54:48,007 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 17:54:48,008 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,010 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,011 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 17:54:48,015 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:48,015 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:48,015 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:48,017 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/recovered.edits] 2023-02-13 17:54:48,017 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/recovered.edits] 2023-02-13 17:54:48,017 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/recovered.edits] 2023-02-13 17:54:48,020 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/180726ec8b01488ca47acf8473048747 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf1/180726ec8b01488ca47acf8473048747 2023-02-13 17:54:48,020 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/61727d04f0164bd09c492ec63cf7eb9c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf1/61727d04f0164bd09c492ec63cf7eb9c 2023-02-13 17:54:48,020 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/036adfc806f34c21acf58f1c0e84e4e4 2023-02-13 17:54:48,027 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0a415759c9684a39a2db5eb2667648b2 2023-02-13 17:54:48,028 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/1fc1628f41784f59ad07457a99b40da8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/cf2/1fc1628f41784f59ad07457a99b40da8 2023-02-13 17:54:48,028 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/70ccb7f180284f229f7adc5d5271bb44 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/cf2/70ccb7f180284f229f7adc5d5271bb44 2023-02-13 17:54:48,028 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/0dbc2409ee1b44f590fcffcf147bffe8 2023-02-13 17:54:48,029 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/332b1dcc2c074a2c8ce8a69f627fb7fc 2023-02-13 17:54:48,030 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/recovered.edits/113.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a/recovered.edits/113.seqid 2023-02-13 17:54:48,031 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/recovered.edits/113.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d/recovered.edits/113.seqid 2023-02-13 17:54:48,031 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/655e2da01c194b66b2f3ff791d4864d8 2023-02-13 17:54:48,031 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:54:48,031 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/db35248b07d43ec3d6842233230b451a 2023-02-13 17:54:48,032 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/6b2d7ada625c443d85d9fb62d35995e7 2023-02-13 17:54:48,033 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/8c75eaa78df44a22af0d9441d98c0db8 2023-02-13 17:54:48,034 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/9038a883dd904e3d975176310bb4c01f 2023-02-13 17:54:48,035 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/a52ff3b1f14041d6b31e6402d4a25cd3 2023-02-13 17:54:48,036 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf1/e0dc27e44c6642cb94aea8bc4f050981 2023-02-13 17:54:48,038 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/00844ebbde6b4b06815bf7f66b8d3922 2023-02-13 17:54:48,039 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/14efa4fa35df4c1e9d07a357c465cc20 2023-02-13 17:54:48,040 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/5ffa2c5e7ac242b0b97ea8dea04445ca 2023-02-13 17:54:48,041 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/6aaaca660ed34746940ed504690bd286 2023-02-13 17:54:48,042 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/76add9fe356344fdac5230f0be99e394 2023-02-13 17:54:48,043 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/7b585d86dd384599a9ec5382355a4546 2023-02-13 17:54:48,044 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/97f857b13e56474aa93439f95348c364 2023-02-13 17:54:48,045 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/cd20a0c73e2e402d89d066b10dbc1ca1 2023-02-13 17:54:48,046 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/ee3b7b2fb97d48f5b012ef975d842862 2023-02-13 17:54:48,047 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/cf2/fe38f26fae7941b69b9c00e473bed2c4 2023-02-13 17:54:48,049 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/recovered.edits/106.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece/recovered.edits/106.seqid 2023-02-13 17:54:48,050 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/22d2fa7f26f9d3723b00490163024ece 2023-02-13 17:54:48,050 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 17:54:48,052 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,054 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 17:54:48,055 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 17:54:48,056 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,056 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 17:54:48,056 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310888056"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:48,056 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310888056"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:48,056 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310888056"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:48,058 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:54:48,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 22d2fa7f26f9d3723b00490163024ece, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676310884374.22d2fa7f26f9d3723b00490163024ece.', STARTKEY => '', ENDKEY => ''}, {ENCODED => db35248b07d43ec3d6842233230b451a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676310886536.db35248b07d43ec3d6842233230b451a.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 5bd57f0845d56cf0851dc8b161781e6d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676310886536.5bd57f0845d56cf0851dc8b161781e6d.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 17:54:48,058 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 17:54:48,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310888058"}]},"ts":"9223372036854775807"} 2023-02-13 17:54:48,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 17:54:48,061 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:54:48,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 53 msec 2023-02-13 17:54:48,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 17:54:48,112 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 17:54:48,127 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=405 (was 400) Potentially hanging thread: hconnection-0x883aab4-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42324 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42056 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42022 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57652 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:45842 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:54222 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:45866 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:45890 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57608 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:46598 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42446 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:54234 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42092 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42098 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:42384 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754918435_15 at /127.0.0.1:57810 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x883aab4-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=848 (was 846) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=217 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=1060 (was 1131) 2023-02-13 17:54:48,141 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=405, OpenFileDescriptor=848, MaxFileDescriptor=60000, SystemLoadAverage=217, ProcessCount=168, AvailableMemoryMB=1060 2023-02-13 17:54:48,141 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:54:48,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:54:48,144 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:54:48,145 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:54:48,147 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:54:48,148 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:54:48,149 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,150 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e empty. 2023-02-13 17:54:48,150 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 17:54:48,159 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:48,160 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => f38e847ef463413e4b3d7445cb3aeb9e, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing f38e847ef463413e4b3d7445cb3aeb9e, disabling compactions & flushes 2023-02-13 17:54:48,167 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. after waiting 0 ms 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,167 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,167 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for f38e847ef463413e4b3d7445cb3aeb9e: Waiting for close lock at 1676310888167Disabling compacts and flushes for region at 1676310888167Disabling writes for close at 1676310888167Writing region close event to WAL at 1676310888167Closed at 1676310888167 2023-02-13 17:54:48,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:54:48,169 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310888169"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310888169"}]},"ts":"1676310888169"} 2023-02-13 17:54:48,171 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:54:48,171 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:54:48,171 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310888171"}]},"ts":"1676310888171"} 2023-02-13 17:54:48,172 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 17:54:48,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:48,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:48,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:48,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:48,175 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:54:48,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, ASSIGN}] 2023-02-13 17:54:48,177 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, ASSIGN 2023-02-13 17:54:48,177 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:54:48,328 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:48,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:48,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310888330"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310888330"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310888330"}]},"ts":"1676310888330"} 2023-02-13 17:54:48,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:48,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f38e847ef463413e4b3d7445cb3aeb9e, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:48,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:48,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:48,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,487 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,488 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,489 DEBUG [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:54:48,490 DEBUG [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:54:48,490 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f38e847ef463413e4b3d7445cb3aeb9e columnFamilyName cf1 2023-02-13 17:54:48,490 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] regionserver.HStore(310): Store=f38e847ef463413e4b3d7445cb3aeb9e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:48,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:54:48,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f38e847ef463413e4b3d7445cb3aeb9e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66724100, jitterRate=-0.0057334303855896}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 17:54:48,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f38e847ef463413e4b3d7445cb3aeb9e: Running coprocessor pre-open hook at 1676310888487Writing region info on filesystem at 1676310888487Initializing all the Stores at 1676310888488 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310888488Cleaning up temporary data from old regions at 1676310888492 (+4 ms)Cleaning up detritus from prior splits at 1676310888493 (+1 ms)Running coprocessor post-open hooks at 1676310888497 (+4 ms)Region opened successfully at 1676310888497 2023-02-13 17:54:48,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e., pid=87, masterSystemTime=1676310888483 2023-02-13 17:54:48,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,501 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:48,501 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310888501"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310888501"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310888501"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310888501"}]},"ts":"1676310888501"} 2023-02-13 17:54:48,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 17:54:48,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699 in 170 msec 2023-02-13 17:54:48,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 17:54:48,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, ASSIGN in 328 msec 2023-02-13 17:54:48,506 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:54:48,506 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310888506"}]},"ts":"1676310888506"} 2023-02-13 17:54:48,507 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 17:54:48,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:54:48,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 368 msec 2023-02-13 17:54:48,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 17:54:48,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:54:48,677 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 17:54:48,678 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 17:54:48,678 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 17:54:48,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 17:54:48,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, REOPEN/MOVE}] 2023-02-13 17:54:48,694 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, REOPEN/MOVE 2023-02-13 17:54:48,694 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:48,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310888694"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310888694"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310888694"}]},"ts":"1676310888694"} 2023-02-13 17:54:48,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:48,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f38e847ef463413e4b3d7445cb3aeb9e, disabling compactions & flushes 2023-02-13 17:54:48,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,848 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. after waiting 0 ms 2023-02-13 17:54:48,849 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 17:54:48,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:54:48,852 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:48,852 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f38e847ef463413e4b3d7445cb3aeb9e: Waiting for close lock at 1676310888847Running coprocessor pre-close hooks at 1676310888847Disabling compacts and flushes for region at 1676310888847Disabling writes for close at 1676310888849 (+2 ms)Writing region close event to WAL at 1676310888849Running coprocessor post-close hooks at 1676310888852 (+3 ms)Closed at 1676310888852 2023-02-13 17:54:48,852 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: f38e847ef463413e4b3d7445cb3aeb9e to self. 2023-02-13 17:54:48,853 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:48,854 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=CLOSED 2023-02-13 17:54:48,854 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310888854"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310888854"}]},"ts":"1676310888854"} 2023-02-13 17:54:48,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 17:54:48,856 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699 in 160 msec 2023-02-13 17:54:48,857 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=true 2023-02-13 17:54:49,007 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:54:49,007 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:49,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310889007"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310889007"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310889007"}]},"ts":"1676310889007"} 2023-02-13 17:54:49,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:54:49,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:49,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f38e847ef463413e4b3d7445cb3aeb9e, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:54:49,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:49,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:49,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,165 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,166 DEBUG [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:54:49,166 DEBUG [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:54:49,166 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f38e847ef463413e4b3d7445cb3aeb9e columnFamilyName cf1 2023-02-13 17:54:49,166 INFO [StoreOpener-f38e847ef463413e4b3d7445cb3aeb9e-1] regionserver.HStore(310): Store=f38e847ef463413e4b3d7445cb3aeb9e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:54:49,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:54:49,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f38e847ef463413e4b3d7445cb3aeb9e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60221299, jitterRate=-0.10263271629810333}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 17:54:49,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f38e847ef463413e4b3d7445cb3aeb9e: Running coprocessor pre-open hook at 1676310889164Writing region info on filesystem at 1676310889164Initializing all the Stores at 1676310889165 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310889165Cleaning up temporary data from old regions at 1676310889168 (+3 ms)Cleaning up detritus from prior splits at 1676310889168Running coprocessor post-open hooks at 1676310889171 (+3 ms)Region opened successfully at 1676310889171 2023-02-13 17:54:49,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e., pid=92, masterSystemTime=1676310889161 2023-02-13 17:54:49,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:49,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:54:49,173 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:54:49,173 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310889173"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310889173"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310889173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310889173"}]},"ts":"1676310889173"} 2023-02-13 17:54:49,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 17:54:49,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699 in 165 msec 2023-02-13 17:54:49,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 17:54:49,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, REOPEN/MOVE in 483 msec 2023-02-13 17:54:49,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 17:54:49,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 497 msec 2023-02-13 17:54:49,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, ASSIGN}] 2023-02-13 17:54:49,183 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, ASSIGN 2023-02-13 17:54:49,184 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 17:54:49,334 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:54:49,335 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:54:49,335 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:54:49,335 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:54:49,335 DEBUG [jenkins-hbase4:33435] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 17:54:49,336 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=879f23954ee5dd3e3e7b467a798bb739, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:54:49,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310889336"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676310889336"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676310889336"}]},"ts":"1676310889336"} 2023-02-13 17:54:49,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 879f23954ee5dd3e3e7b467a798bb739, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:54:49,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:54:49,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 879f23954ee5dd3e3e7b467a798bb739, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 17:54:49,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:54:49,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:54:49,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:54:49,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:54:49,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:54:49,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:49,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:49,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:49,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:49,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:49,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:50,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:51,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,236 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:54:52,279 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 17:54:52,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:52,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:53,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:54,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:55,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:56,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:57,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:58,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:54:59,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:00,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:01,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:02,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:03,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:04,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:05,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:06,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:07,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:08,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:09,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:10,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:11,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:12,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:13,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:14,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:15,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:16,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:17,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:18,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:19,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:20,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:21,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,113 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 17:55:22,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,168 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 17:55:22,183 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 17:55:22,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:22,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:23,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:24,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:25,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:26,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:27,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:28,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:29,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:30,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:31,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:32,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:33,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:34,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:35,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:36,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:37,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:38,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:39,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:40,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:41,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:42,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:43,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:44,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:45,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:46,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:47,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 17:55:48,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:48,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 17:55:48,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 17:55:49,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 17:55:49,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:49,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 17:55:49,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:50,880 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:50,881 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:50,882 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:50,882 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:50,882 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:50,884 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting f38e847ef463413e4b3d7445cb3aeb9e, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:50,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c exec-time=6 msec 2023-02-13 17:55:50,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:51,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:51,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 17:55:51,216 INFO [StoreOpener-879f23954ee5dd3e3e7b467a798bb739-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:55:51,220 DEBUG [StoreOpener-879f23954ee5dd3e3e7b467a798bb739-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:55:51,220 DEBUG [StoreOpener-879f23954ee5dd3e3e7b467a798bb739-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1 2023-02-13 17:55:51,221 INFO [StoreOpener-879f23954ee5dd3e3e7b467a798bb739-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 879f23954ee5dd3e3e7b467a798bb739 columnFamilyName cf1 2023-02-13 17:55:51,221 INFO [StoreOpener-879f23954ee5dd3e3e7b467a798bb739-1] regionserver.HStore(310): Store=879f23954ee5dd3e3e7b467a798bb739/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:55:51,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 879f23954ee5dd3e3e7b467a798bb739; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59384046, jitterRate=-0.11510875821113586}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 17:55:51,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 879f23954ee5dd3e3e7b467a798bb739: Running coprocessor pre-open hook at 1676310889495Initializing all the Stores at 1676310951215 (+61720 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310951215Running coprocessor post-open hooks at 1676310951223 (+8 ms)Region opened successfully at 1676310951223 2023-02-13 17:55:51,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739., pid=94, masterSystemTime=1676310889492 2023-02-13 17:55:51,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:51,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:51,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=879f23954ee5dd3e3e7b467a798bb739, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:51,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310951225"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676310951225"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676310951225"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676310951225"}]},"ts":"1676310951225"} 2023-02-13 17:55:51,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 17:55:51,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 879f23954ee5dd3e3e7b467a798bb739, server=jenkins-hbase4.apache.org,39639,1676310860746 in 1 mins, 1.888 sec 2023-02-13 17:55:51,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 17:55:51,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, ASSIGN in 1 mins, 2.046 sec 2023-02-13 17:55:51,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.57 sec 2023-02-13 17:55:51,232 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676310888653","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62578,"client":"172.31.14.131:44722","queuetimems":0,"class":"HMaster"} 2023-02-13 17:55:51,232 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 605, skipping over this response of 3 bytes 2023-02-13 17:55:53,132 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up 2023-02-13 17:55:53,133 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:55:53,134 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,135 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 17:55:53,138 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310953138"}]},"ts":"1676310953138"} 2023-02-13 17:55:53,139 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 17:55:53,140 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 17:55:53,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, UNASSIGN}] 2023-02-13 17:55:53,141 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, UNASSIGN 2023-02-13 17:55:53,142 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=879f23954ee5dd3e3e7b467a798bb739, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:53,142 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310953142"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676310953142"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676310953142"}]},"ts":"1676310953142"} 2023-02-13 17:55:53,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 879f23954ee5dd3e3e7b467a798bb739, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:55:53,144 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, UNASSIGN 2023-02-13 17:55:53,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:53,145 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310953145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310953145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310953145"}]},"ts":"1676310953145"} 2023-02-13 17:55:53,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:55:53,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 17:55:53,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:55:53,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 879f23954ee5dd3e3e7b467a798bb739, disabling compactions & flushes 2023-02-13 17:55:53,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:53,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:53,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. after waiting 0 ms 2023-02-13 17:55:53,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:53,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:55:53,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739. 2023-02-13 17:55:53,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 879f23954ee5dd3e3e7b467a798bb739: Waiting for close lock at 1676310953295Running coprocessor pre-close hooks at 1676310953295Disabling compacts and flushes for region at 1676310953295Disabling writes for close at 1676310953296 (+1 ms)Writing region close event to WAL at 1676310953296Running coprocessor post-close hooks at 1676310953296Closed at 1676310953297 (+1 ms) 2023-02-13 17:55:53,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:55:53,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f38e847ef463413e4b3d7445cb3aeb9e, disabling compactions & flushes 2023-02-13 17:55:53,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:55:53,298 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 879f23954ee5dd3e3e7b467a798bb739 2023-02-13 17:55:53,298 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:55:53,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. after waiting 0 ms 2023-02-13 17:55:53,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:55:53,299 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=879f23954ee5dd3e3e7b467a798bb739, regionState=CLOSED 2023-02-13 17:55:53,299 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310953299"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676310953299"}]},"ts":"1676310953299"} 2023-02-13 17:55:53,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 17:55:53,303 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 879f23954ee5dd3e3e7b467a798bb739, server=jenkins-hbase4.apache.org,39639,1676310860746 in 158 msec 2023-02-13 17:55:53,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 17:55:53,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:55:53,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e. 2023-02-13 17:55:53,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f38e847ef463413e4b3d7445cb3aeb9e: Waiting for close lock at 1676310953298Running coprocessor pre-close hooks at 1676310953298Disabling compacts and flushes for region at 1676310953298Disabling writes for close at 1676310953299 (+1 ms)Writing region close event to WAL at 1676310953299Running coprocessor post-close hooks at 1676310953303 (+4 ms)Closed at 1676310953304 (+1 ms) 2023-02-13 17:55:53,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739, UNASSIGN in 162 msec 2023-02-13 17:55:53,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:55:53,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=f38e847ef463413e4b3d7445cb3aeb9e, regionState=CLOSED 2023-02-13 17:55:53,306 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676310953305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310953305"}]},"ts":"1676310953305"} 2023-02-13 17:55:53,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 17:55:53,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure f38e847ef463413e4b3d7445cb3aeb9e, server=jenkins-hbase4.apache.org,46577,1676310860699 in 161 msec 2023-02-13 17:55:53,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 17:55:53,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=f38e847ef463413e4b3d7445cb3aeb9e, UNASSIGN in 167 msec 2023-02-13 17:55:53,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310953311"}]},"ts":"1676310953311"} 2023-02-13 17:55:53,312 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 17:55:53,314 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 17:55:53,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 180 msec 2023-02-13 17:55:53,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 17:55:53,439 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 17:55:53,440 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,442 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 17:55:53,447 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:55:53,448 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits] 2023-02-13 17:55:53,452 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits/7.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e/recovered.edits/7.seqid 2023-02-13 17:55:53,452 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/f38e847ef463413e4b3d7445cb3aeb9e 2023-02-13 17:55:53,452 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 17:55:53,454 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,456 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 17:55:53,458 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 17:55:53,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,459 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 17:55:53,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310953459"}]},"ts":"9223372036854775807"} 2023-02-13 17:55:53,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310953459"}]},"ts":"9223372036854775807"} 2023-02-13 17:55:53,460 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 17:55:53,460 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f38e847ef463413e4b3d7445cb3aeb9e, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145.f38e847ef463413e4b3d7445cb3aeb9e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 879f23954ee5dd3e3e7b467a798bb739, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676310888145_0001.879f23954ee5dd3e3e7b467a798bb739.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 17:55:53,460 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 17:55:53,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310953460"}]},"ts":"9223372036854775807"} 2023-02-13 17:55:53,462 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 17:55:53,463 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:53,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 23 msec 2023-02-13 17:55:53,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 17:55:53,544 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 17:55:53,556 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=341 (was 405), OpenFileDescriptor=652 (was 848), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=132 (was 217), ProcessCount=168 (was 168), AvailableMemoryMB=915 (was 1060) 2023-02-13 17:55:53,568 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=342, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=132, ProcessCount=168, AvailableMemoryMB=914 2023-02-13 17:55:53,569 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:55:53,571 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:55:53,571 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:55:53,572 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:53,573 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:55:53,574 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:55:53,576 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,576 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff empty. 2023-02-13 17:55:53,577 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,577 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 17:55:53,586 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 17:55:53,587 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => f9cb39e998cf42cdc3088fafbd3631ff, NAME => 'testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing f9cb39e998cf42cdc3088fafbd3631ff, disabling compactions & flushes 2023-02-13 17:55:53,593 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. after waiting 0 ms 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,593 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,593 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for f9cb39e998cf42cdc3088fafbd3631ff: Waiting for close lock at 1676310953593Disabling compacts and flushes for region at 1676310953593Disabling writes for close at 1676310953593Writing region close event to WAL at 1676310953593Closed at 1676310953593 2023-02-13 17:55:53,595 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:55:53,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676310953595"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310953595"}]},"ts":"1676310953595"} 2023-02-13 17:55:53,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:55:53,597 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:55:53,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310953597"}]},"ts":"1676310953597"} 2023-02-13 17:55:53,598 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 17:55:53,602 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:55:53,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:55:53,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:55:53,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:55:53,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:55:53,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, ASSIGN}] 2023-02-13 17:55:53,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, ASSIGN 2023-02-13 17:55:53,605 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:55:53,755 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:55:53,756 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=f9cb39e998cf42cdc3088fafbd3631ff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:53,756 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676310953756"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310953756"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310953756"}]},"ts":"1676310953756"} 2023-02-13 17:55:53,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure f9cb39e998cf42cdc3088fafbd3631ff, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:55:53,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,912 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f9cb39e998cf42cdc3088fafbd3631ff, NAME => 'testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:55:53,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:55:53,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:55:53,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,913 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,914 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,915 DEBUG [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1 2023-02-13 17:55:53,915 DEBUG [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1 2023-02-13 17:55:53,915 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9cb39e998cf42cdc3088fafbd3631ff columnFamilyName cf1 2023-02-13 17:55:53,916 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] regionserver.HStore(310): Store=f9cb39e998cf42cdc3088fafbd3631ff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:55:53,916 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,917 DEBUG [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2 2023-02-13 17:55:53,917 DEBUG [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2 2023-02-13 17:55:53,917 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9cb39e998cf42cdc3088fafbd3631ff columnFamilyName cf2 2023-02-13 17:55:53,918 INFO [StoreOpener-f9cb39e998cf42cdc3088fafbd3631ff-1] regionserver.HStore(310): Store=f9cb39e998cf42cdc3088fafbd3631ff/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:55:53,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:55:53,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:53,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:55:53,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f9cb39e998cf42cdc3088fafbd3631ff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73474896, jitterRate=0.09486126899719238}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:55:53,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f9cb39e998cf42cdc3088fafbd3631ff: Running coprocessor pre-open hook at 1676310953913Writing region info on filesystem at 1676310953913Initializing all the Stores at 1676310953913Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310953913Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310953914 (+1 ms)Cleaning up temporary data from old regions at 1676310953919 (+5 ms)Cleaning up detritus from prior splits at 1676310953919Running coprocessor post-open hooks at 1676310953922 (+3 ms)Region opened successfully at 1676310953923 (+1 ms) 2023-02-13 17:55:53,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff., pid=104, masterSystemTime=1676310953909 2023-02-13 17:55:53,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:53,925 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=f9cb39e998cf42cdc3088fafbd3631ff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:53,925 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676310953925"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310953925"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310953925"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310953925"}]},"ts":"1676310953925"} 2023-02-13 17:55:53,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 17:55:53,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure f9cb39e998cf42cdc3088fafbd3631ff, server=jenkins-hbase4.apache.org,46577,1676310860699 in 168 msec 2023-02-13 17:55:53,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 17:55:53,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, ASSIGN in 325 msec 2023-02-13 17:55:53,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:55:53,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310953930"}]},"ts":"1676310953930"} 2023-02-13 17:55:53,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 17:55:53,932 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:55:53,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 361 msec 2023-02-13 17:55:54,079 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,081 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,081 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,081 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,081 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,081 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,081 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,082 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,082 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,083 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,083 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,084 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,084 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,085 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:55:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,095 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933 2023-02-13 17:55:54,110 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3 2023-02-13 17:55:54,114 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933 2023-02-13 17:55:54,119 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:55:54,119 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3 2023-02-13 17:55:54,123 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:55:54,124 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 39ms, sequenceid=6, compaction requested=false 2023-02-13 17:55:54,124 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 17:55:54,124 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,124 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,126 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,126 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,128 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d03797e[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,129 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,130 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,129 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,132 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,135 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,136 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,136 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,136 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,136 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,137 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,141 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:54,143 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,143 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,143 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,144 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,244 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:54,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:54,255 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:54,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,257 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,257 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,258 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,258 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,258 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,259 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,259 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,260 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,260 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,260 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,261 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,261 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,261 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,270 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/fdeb9105a34f4356b3232661ee87b8a1 2023-02-13 17:55:54,280 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/f1a4620337dd4920a2db2fc7e7d273e8 2023-02-13 17:55:54,284 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/fdeb9105a34f4356b3232661ee87b8a1 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1 2023-02-13 17:55:54,288 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:55:54,289 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/f1a4620337dd4920a2db2fc7e7d273e8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8 2023-02-13 17:55:54,293 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:55:54,294 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 33ms, sequenceid=19, compaction requested=false 2023-02-13 17:55:54,294 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,294 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,296 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,296 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,297 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,297 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@307f680c[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,299 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,299 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,300 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,300 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,300 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,300 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,300 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,301 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,302 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,304 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,304 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,304 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,304 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,308 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,308 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,308 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,310 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,311 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,311 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,311 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,313 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,314 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,314 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,317 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,317 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:54,317 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,317 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,318 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,418 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:54,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:54,430 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:54,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,432 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,432 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,433 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,433 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,435 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,436 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,436 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,436 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,436 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:54,438 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,439 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,444 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/6a6d9a7a912949beb43d2986c78d8bf3 2023-02-13 17:55:54,454 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/9d220bfbf2d6452ba509cbb1dd08e23c 2023-02-13 17:55:54,457 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/6a6d9a7a912949beb43d2986c78d8bf3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/6a6d9a7a912949beb43d2986c78d8bf3 2023-02-13 17:55:54,460 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/6a6d9a7a912949beb43d2986c78d8bf3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:55:54,461 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/9d220bfbf2d6452ba509cbb1dd08e23c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9d220bfbf2d6452ba509cbb1dd08e23c 2023-02-13 17:55:54,464 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9d220bfbf2d6452ba509cbb1dd08e23c, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:55:54,465 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 29ms, sequenceid=32, compaction requested=true 2023-02-13 17:55:54,465 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,465 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,465 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,467 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,467 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,467 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,467 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16e5a1f4[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,469 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,469 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,470 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,470 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,471 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,476 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,476 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,476 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,476 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,477 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,477 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,478 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,479 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,479 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,490 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,489 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,490 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:54,490 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:54,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:54,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:54,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,490 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,491 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,591 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:54,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:54,603 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:54,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,604 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,605 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,606 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,606 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,607 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,607 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,607 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,608 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,610 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,610 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,610 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:54,618 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/4303412f33f14909a2693959716d16c0 2023-02-13 17:55:54,629 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/92738bc4831b4e1784e3f27ec84bc7a4 2023-02-13 17:55:54,632 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/4303412f33f14909a2693959716d16c0 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/4303412f33f14909a2693959716d16c0 2023-02-13 17:55:54,636 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/4303412f33f14909a2693959716d16c0, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:55:54,636 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/92738bc4831b4e1784e3f27ec84bc7a4 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/92738bc4831b4e1784e3f27ec84bc7a4 2023-02-13 17:55:54,639 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/92738bc4831b4e1784e3f27ec84bc7a4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:55:54,640 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 30ms, sequenceid=45, compaction requested=true 2023-02-13 17:55:54,640 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,640 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,640 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,640 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,640 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,640 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,640 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,643 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,643 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,643 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,643 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4e70392d[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,645 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,645 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,646 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,646 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,648 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,651 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,651 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,651 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,651 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,651 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,655 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,655 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,655 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,657 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,657 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,658 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,660 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:54,660 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,660 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,661 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,761 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:54,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:54,773 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:54,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,774 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,776 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,776 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,776 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,776 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,777 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,777 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,778 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,778 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,778 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,778 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,778 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,779 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:54,779 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,781 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,781 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,781 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,781 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,781 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,786 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/c4bfc7bc9def48b0b49419e5095aec71 2023-02-13 17:55:54,796 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/cfa80c3966614d9abcf93ec3b6c52e2e 2023-02-13 17:55:54,800 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/c4bfc7bc9def48b0b49419e5095aec71 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/c4bfc7bc9def48b0b49419e5095aec71 2023-02-13 17:55:54,803 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/c4bfc7bc9def48b0b49419e5095aec71, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:55:54,804 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/cfa80c3966614d9abcf93ec3b6c52e2e as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/cfa80c3966614d9abcf93ec3b6c52e2e 2023-02-13 17:55:54,808 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/cfa80c3966614d9abcf93ec3b6c52e2e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:55:54,809 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 31ms, sequenceid=58, compaction requested=true 2023-02-13 17:55:54,809 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,809 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,809 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,810 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,811 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,811 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@788e7114[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,812 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,812 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,813 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,814 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,814 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,814 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,819 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,819 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,819 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,822 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,825 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,825 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,825 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,825 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,825 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,827 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,827 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,828 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,829 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,830 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,830 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,832 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,832 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,832 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:54,833 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:54,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,833 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,934 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:54,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:54,944 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:54,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:54,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 17:55:54,946 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 17:55:54,946 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,946 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 17:55:54,946 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 17:55:54,946 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,946 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,948 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,948 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,948 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:54,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:54,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,949 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 17:55:54,950 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,950 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,950 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,951 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:54,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:54,951 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,951 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. started... 2023-02-13 17:55:54,951 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:54,952 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,952 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,953 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,953 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,953 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,953 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,960 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/5cdf12aec8ad41cd9efa494700686c7b 2023-02-13 17:55:54,970 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/e30982ab57564a3cb7859b00e5779432 2023-02-13 17:55:54,974 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/5cdf12aec8ad41cd9efa494700686c7b as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/5cdf12aec8ad41cd9efa494700686c7b 2023-02-13 17:55:54,977 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/5cdf12aec8ad41cd9efa494700686c7b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:55:54,978 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/e30982ab57564a3cb7859b00e5779432 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/e30982ab57564a3cb7859b00e5779432 2023-02-13 17:55:54,981 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/e30982ab57564a3cb7859b00e5779432, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:55:54,981 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 30ms, sequenceid=71, compaction requested=true 2023-02-13 17:55:54,981 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:54,981 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:54,981 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:54,981 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:54,982 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,982 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 17:55:54,982 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 17:55:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,983 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,983 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 17:55:54,985 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@337a4901[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:54,985 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 17:55:54,985 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,987 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,987 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,987 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,988 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:54,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,990 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:54,990 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 17:55:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:55:54,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 17:55:54,993 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 17:55:54,993 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:54,993 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:54,993 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 17:55:54,993 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,995 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,995 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:54,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:54,996 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,997 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:54,997 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 17:55:54,997 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,000 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:55,001 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 17:55:55,001 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 17:55:55,001 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:55,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:55,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:55,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:55,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:55,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:55,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:55,001 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:55,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:55,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 17:55:55,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 17:55:55,102 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:55,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:55,109 INFO [Listener at localhost/44157] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,110 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.StoreUtils(129): cannot split hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1 because midkey is the same as first or last row 2023-02-13 17:55:55,110 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.StoreUtils(129): cannot split hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8 because midkey is the same as first or last row 2023-02-13 17:55:55,111 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HRegion(2744): Flushing f9cb39e998cf42cdc3088fafbd3631ff 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:55:55,121 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/52c5807b95ab49008614443958b6ce03 2023-02-13 17:55:55,133 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/5e5e9c140fc8476f9e1849dfbb4a05af 2023-02-13 17:55:55,137 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf1/52c5807b95ab49008614443958b6ce03 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/52c5807b95ab49008614443958b6ce03 2023-02-13 17:55:55,140 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/52c5807b95ab49008614443958b6ce03, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:55:55,141 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/.tmp/cf2/5e5e9c140fc8476f9e1849dfbb4a05af as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/5e5e9c140fc8476f9e1849dfbb4a05af 2023-02-13 17:55:55,145 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/5e5e9c140fc8476f9e1849dfbb4a05af, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:55:55,146 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f9cb39e998cf42cdc3088fafbd3631ff in 36ms, sequenceid=82, compaction requested=true 2023-02-13 17:55:55,146 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.HRegion(2445): Flush status journal for f9cb39e998cf42cdc3088fafbd3631ff: 2023-02-13 17:55:55,147 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.StoreUtils(129): cannot split hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1 because midkey is the same as first or last row 2023-02-13 17:55:55,147 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46577] regionserver.StoreUtils(129): cannot split hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8 because midkey is the same as first or last row 2023-02-13 17:55:55,147 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,147 DEBUG [Listener at localhost/44157] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 17:55:55,147 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:55:55,149 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 17:55:55,149 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-13 17:55:55,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 17:55:55,152 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310955152"}]},"ts":"1676310955152"} 2023-02-13 17:55:55,153 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 17:55:55,154 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 17:55:55,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, UNASSIGN}] 2023-02-13 17:55:55,155 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, UNASSIGN 2023-02-13 17:55:55,156 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f9cb39e998cf42cdc3088fafbd3631ff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:55,156 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676310955156"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310955156"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310955156"}]},"ts":"1676310955156"} 2023-02-13 17:55:55,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure f9cb39e998cf42cdc3088fafbd3631ff, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:55:55,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 17:55:55,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:55,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f9cb39e998cf42cdc3088fafbd3631ff, disabling compactions & flushes 2023-02-13 17:55:55,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:55,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:55,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. after waiting 0 ms 2023-02-13 17:55:55,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:55,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:55:55,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:55:55,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff. 2023-02-13 17:55:55,313 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f9cb39e998cf42cdc3088fafbd3631ff: Waiting for close lock at 1676310955309Running coprocessor pre-close hooks at 1676310955309Disabling compacts and flushes for region at 1676310955309Disabling writes for close at 1676310955309Writing region close event to WAL at 1676310955310 (+1 ms)Running coprocessor post-close hooks at 1676310955313 (+3 ms)Closed at 1676310955313 2023-02-13 17:55:55,314 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:55,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f9cb39e998cf42cdc3088fafbd3631ff, regionState=CLOSED 2023-02-13 17:55:55,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676310955314"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310955314"}]},"ts":"1676310955314"} 2023-02-13 17:55:55,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 17:55:55,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure f9cb39e998cf42cdc3088fafbd3631ff, server=jenkins-hbase4.apache.org,46577,1676310860699 in 158 msec 2023-02-13 17:55:55,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 17:55:55,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=f9cb39e998cf42cdc3088fafbd3631ff, UNASSIGN in 162 msec 2023-02-13 17:55:55,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310955318"}]},"ts":"1676310955318"} 2023-02-13 17:55:55,319 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 17:55:55,321 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 17:55:55,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 172 msec 2023-02-13 17:55:55,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 17:55:55,454 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 17:55:55,454 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-13 17:55:55,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 17:55:55,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,460 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:55,462 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/recovered.edits] 2023-02-13 17:55:55,465 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/4303412f33f14909a2693959716d16c0 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/4303412f33f14909a2693959716d16c0 2023-02-13 17:55:55,466 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/52c5807b95ab49008614443958b6ce03 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/52c5807b95ab49008614443958b6ce03 2023-02-13 17:55:55,467 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/5cdf12aec8ad41cd9efa494700686c7b to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/5cdf12aec8ad41cd9efa494700686c7b 2023-02-13 17:55:55,468 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/6a6d9a7a912949beb43d2986c78d8bf3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/6a6d9a7a912949beb43d2986c78d8bf3 2023-02-13 17:55:55,469 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/7e8df5689c294ac9aa9cd3d9d2d1b933 2023-02-13 17:55:55,469 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/c4bfc7bc9def48b0b49419e5095aec71 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/c4bfc7bc9def48b0b49419e5095aec71 2023-02-13 17:55:55,470 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf1/fdeb9105a34f4356b3232661ee87b8a1 2023-02-13 17:55:55,472 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/5e5e9c140fc8476f9e1849dfbb4a05af to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/5e5e9c140fc8476f9e1849dfbb4a05af 2023-02-13 17:55:55,473 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/92738bc4831b4e1784e3f27ec84bc7a4 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/92738bc4831b4e1784e3f27ec84bc7a4 2023-02-13 17:55:55,474 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9d220bfbf2d6452ba509cbb1dd08e23c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9d220bfbf2d6452ba509cbb1dd08e23c 2023-02-13 17:55:55,475 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/9e3c9f11665d4a5ea02b472c6cc2aaa3 2023-02-13 17:55:55,476 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/cfa80c3966614d9abcf93ec3b6c52e2e to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/cfa80c3966614d9abcf93ec3b6c52e2e 2023-02-13 17:55:55,477 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/e30982ab57564a3cb7859b00e5779432 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/e30982ab57564a3cb7859b00e5779432 2023-02-13 17:55:55,477 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/cf2/f1a4620337dd4920a2db2fc7e7d273e8 2023-02-13 17:55:55,479 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff/recovered.edits/85.seqid 2023-02-13 17:55:55,480 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testInvalidSplitKey/f9cb39e998cf42cdc3088fafbd3631ff 2023-02-13 17:55:55,480 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 17:55:55,482 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,483 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 17:55:55,486 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 17:55:55,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 17:55:55,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310955487"}]},"ts":"9223372036854775807"} 2023-02-13 17:55:55,489 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 17:55:55,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f9cb39e998cf42cdc3088fafbd3631ff, NAME => 'testInvalidSplitKey,,1676310953571.f9cb39e998cf42cdc3088fafbd3631ff.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 17:55:55,489 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 17:55:55,489 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310955489"}]},"ts":"9223372036854775807"} 2023-02-13 17:55:55,490 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 17:55:55,492 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:55,493 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 38 msec 2023-02-13 17:55:55,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 17:55:55,558 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 17:55:55,571 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=361 (was 342) - Thread LEAK? -, OpenFileDescriptor=736 (was 652) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=138 (was 132) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=911 (was 914) 2023-02-13 17:55:55,584 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=361, OpenFileDescriptor=736, MaxFileDescriptor=60000, SystemLoadAverage=138, ProcessCount=168, AvailableMemoryMB=911 2023-02-13 17:55:55,584 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:55:55,586 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:55:55,587 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:55:55,588 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:55,589 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:55:55,590 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:55:55,591 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,591 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 empty. 2023-02-13 17:55:55,592 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,592 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 17:55:55,600 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 17:55:55,601 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2ad108a993b4834a8be9722478b2d883, NAME => 'testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:55:55,607 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:55:55,607 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 2ad108a993b4834a8be9722478b2d883, disabling compactions & flushes 2023-02-13 17:55:55,607 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,607 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,607 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. after waiting 0 ms 2023-02-13 17:55:55,607 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,608 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,608 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 2ad108a993b4834a8be9722478b2d883: Waiting for close lock at 1676310955607Disabling compacts and flushes for region at 1676310955607Disabling writes for close at 1676310955607Writing region close event to WAL at 1676310955608 (+1 ms)Closed at 1676310955608 2023-02-13 17:55:55,609 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:55:55,610 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310955610"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310955610"}]},"ts":"1676310955610"} 2023-02-13 17:55:55,611 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:55:55,611 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:55:55,612 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310955611"}]},"ts":"1676310955611"} 2023-02-13 17:55:55,612 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 17:55:55,615 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:55:55,615 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:55:55,615 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:55:55,615 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:55:55,615 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:55:55,616 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, ASSIGN}] 2023-02-13 17:55:55,617 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, ASSIGN 2023-02-13 17:55:55,617 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34255,1676310860648; forceNewPlan=false, retain=false 2023-02-13 17:55:55,767 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:55:55,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2ad108a993b4834a8be9722478b2d883, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:55,769 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310955769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310955769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310955769"}]},"ts":"1676310955769"} 2023-02-13 17:55:55,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:55:55,922 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:55,923 DEBUG [RSProcedureDispatcher-pool-4] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 17:55:55,925 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59758, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 17:55:55,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2ad108a993b4834a8be9722478b2d883, NAME => 'testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:55:55,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:55:55,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:55:55,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,930 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,932 DEBUG [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1 2023-02-13 17:55:55,932 DEBUG [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1 2023-02-13 17:55:55,932 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ad108a993b4834a8be9722478b2d883 columnFamilyName cf1 2023-02-13 17:55:55,932 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] regionserver.HStore(310): Store=2ad108a993b4834a8be9722478b2d883/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:55:55,933 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,934 DEBUG [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2 2023-02-13 17:55:55,934 DEBUG [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2 2023-02-13 17:55:55,934 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ad108a993b4834a8be9722478b2d883 columnFamilyName cf2 2023-02-13 17:55:55,934 INFO [StoreOpener-2ad108a993b4834a8be9722478b2d883-1] regionserver.HStore(310): Store=2ad108a993b4834a8be9722478b2d883/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:55:55,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:55:55,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:55,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:55:55,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2ad108a993b4834a8be9722478b2d883; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72172649, jitterRate=0.07545627653598785}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:55:55,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2ad108a993b4834a8be9722478b2d883: Running coprocessor pre-open hook at 1676310955930Writing region info on filesystem at 1676310955930Initializing all the Stores at 1676310955930Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310955930Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310955930Cleaning up temporary data from old regions at 1676310955936 (+6 ms)Cleaning up detritus from prior splits at 1676310955936Running coprocessor post-open hooks at 1676310955940 (+4 ms)Region opened successfully at 1676310955940 2023-02-13 17:55:55,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883., pid=111, masterSystemTime=1676310955922 2023-02-13 17:55:55,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:55,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=2ad108a993b4834a8be9722478b2d883, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:55,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310955944"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310955944"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310955944"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310955944"}]},"ts":"1676310955944"} 2023-02-13 17:55:55,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 17:55:55,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 in 176 msec 2023-02-13 17:55:55,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 17:55:55,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, ASSIGN in 336 msec 2023-02-13 17:55:55,954 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:55:55,955 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310955955"}]},"ts":"1676310955955"} 2023-02-13 17:55:55,956 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 17:55:55,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:55:55,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 371 msec 2023-02-13 17:55:56,090 DEBUG [Listener at localhost/44157] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 17:55:56,094 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59768, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 17:55:56,099 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,100 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,100 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,100 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,100 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,101 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,101 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,102 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,102 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,102 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,102 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,102 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,103 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,103 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,104 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,104 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,105 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:55:56,105 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,105 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,105 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,106 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,106 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,106 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,113 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/99160cfab74648399fdddbbea98695ac 2023-02-13 17:55:56,124 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/b882c9709fbe446d96d5d9abdca229a7 2023-02-13 17:55:56,127 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/99160cfab74648399fdddbbea98695ac as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac 2023-02-13 17:55:56,131 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:55:56,131 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/b882c9709fbe446d96d5d9abdca229a7 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 2023-02-13 17:55:56,134 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:55:56,135 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 30ms, sequenceid=6, compaction requested=false 2023-02-13 17:55:56,135 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,135 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,136 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,136 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,136 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,136 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,136 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,136 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,138 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,138 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,138 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,138 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8abb49b[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,141 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,142 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,142 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,143 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,143 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,143 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,143 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,144 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:56,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:56,147 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:56,147 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:56,147 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:56,147 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,147 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,150 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,150 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,151 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,151 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,153 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,153 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,153 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,155 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,156 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,156 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,156 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,157 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,157 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,157 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,157 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,256 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:56,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:56,267 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:56,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,268 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,268 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,268 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,268 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,269 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,269 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,270 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,271 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,271 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,273 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,273 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,273 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,273 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:56,274 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,275 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,281 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/2550fec6a72945508c0ca3bcdaa720c9 2023-02-13 17:55:56,290 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c 2023-02-13 17:55:56,294 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/2550fec6a72945508c0ca3bcdaa720c9 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 2023-02-13 17:55:56,297 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:55:56,297 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c 2023-02-13 17:55:56,301 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:55:56,301 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 28ms, sequenceid=19, compaction requested=false 2023-02-13 17:55:56,301 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,301 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,301 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,301 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,302 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,305 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,305 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,305 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47dd4df2[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,307 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,307 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,308 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,308 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,309 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,309 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,309 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,309 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,311 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:56,314 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:56,314 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:56,314 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:56,314 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:56,314 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,314 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,315 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,316 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,316 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,316 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,318 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,318 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,318 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,321 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,321 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,321 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:56,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,321 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,322 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,422 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:56,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:56,433 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:56,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,434 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,434 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,435 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,436 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,436 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,436 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,436 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,437 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,437 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,437 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,437 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,437 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,438 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,438 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,438 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,439 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,439 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,439 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,439 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,439 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:56,440 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,441 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,441 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,449 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/78cf0b64627a457b8e24bdc678e9c369 2023-02-13 17:55:56,459 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/aba94b15b2fe446f8b8e53cdc90776e2 2023-02-13 17:55:56,462 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/78cf0b64627a457b8e24bdc678e9c369 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 2023-02-13 17:55:56,465 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:55:56,466 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/aba94b15b2fe446f8b8e53cdc90776e2 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 2023-02-13 17:55:56,469 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:55:56,469 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 30ms, sequenceid=32, compaction requested=true 2023-02-13 17:55:56,469 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,469 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,469 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,470 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,471 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,472 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cd092cc[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,473 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,473 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,474 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,474 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,475 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,475 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,477 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,478 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,478 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:56,480 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:56,480 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:56,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:56,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:56,481 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,481 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,483 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,483 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,484 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,485 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,485 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,486 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,492 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,492 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,492 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,492 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,493 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,593 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:56,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:56,604 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:56,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,606 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,606 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,607 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,609 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,609 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,609 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,610 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,610 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,610 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,610 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:56,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,611 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,612 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,612 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,612 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,618 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/2d18a6a534f140be834f4ca3eb4beae6 2023-02-13 17:55:56,630 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 2023-02-13 17:55:56,634 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/2d18a6a534f140be834f4ca3eb4beae6 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 2023-02-13 17:55:56,637 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:55:56,637 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 2023-02-13 17:55:56,640 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:55:56,641 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 31ms, sequenceid=45, compaction requested=true 2023-02-13 17:55:56,641 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,641 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,641 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,641 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,641 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,641 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,641 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,644 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,644 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,644 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,644 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d83e816[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,646 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,646 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,647 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,647 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,648 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,649 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,649 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:56,652 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:56,652 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:56,652 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:56,652 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:56,652 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,652 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,654 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,655 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,655 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,656 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,657 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,657 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,657 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,660 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:56,660 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,660 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:56,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,664 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,761 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:56,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:56,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:56,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,773 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,774 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,775 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,775 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,775 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,776 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,777 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,777 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,777 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,777 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,777 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,777 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,777 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:56,778 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,780 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,780 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,780 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,781 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,781 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,787 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/3c921158225d4063b857a4f3036e7353 2023-02-13 17:55:56,800 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/715f98e6f39945f6b965f895e41819a9 2023-02-13 17:55:56,804 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/3c921158225d4063b857a4f3036e7353 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 2023-02-13 17:55:56,808 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:55:56,808 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/715f98e6f39945f6b965f895e41819a9 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 2023-02-13 17:55:56,812 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:55:56,813 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 36ms, sequenceid=58, compaction requested=true 2023-02-13 17:55:56,813 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,813 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,813 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,813 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,813 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,813 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,813 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,818 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,818 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,820 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78598e51[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,820 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,820 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,822 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,822 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,824 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,824 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,824 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,824 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,825 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,824 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,825 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,825 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,825 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,825 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:56,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:56,827 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:56,827 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:56,827 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:56,827 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,827 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,830 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,830 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,831 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,832 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,832 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,834 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:56,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,834 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,834 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,835 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:56,835 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,836 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,836 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,935 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:56,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:56,947 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:55:56,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 17:55:56,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 17:55:56,948 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:56,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 17:55:56,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 17:55:56,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,949 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,956 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,956 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:55:56,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:56,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,957 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 17:55:56,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,958 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 17:55:56,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,958 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:55:56,958 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,958 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,958 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. started... 2023-02-13 17:55:56,958 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,960 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,966 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/58592fd9d025457a8817194dea55c357 2023-02-13 17:55:56,976 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/e83a1858084c45e08fb24dc03f3d540d 2023-02-13 17:55:56,980 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/58592fd9d025457a8817194dea55c357 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 2023-02-13 17:55:56,984 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:55:56,985 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/e83a1858084c45e08fb24dc03f3d540d as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d 2023-02-13 17:55:56,988 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:55:56,989 INFO [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 31ms, sequenceid=71, compaction requested=true 2023-02-13 17:55:56,989 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2ad108a993b4834a8be9722478b2d883: 2023-02-13 17:55:56,989 DEBUG [rs(jenkins-hbase4.apache.org,34255,1676310860648)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:55:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:55:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 17:55:56,989 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 17:55:56,990 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,990 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,991 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,991 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:55:56,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 17:55:56,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5546fe75[Count = 0] remaining members to acquire global barrier 2023-02-13 17:55:56,993 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 17:55:56,993 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,994 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:55:56,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:55:56,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 17:55:56,994 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:56,997 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,997 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 17:55:56,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:56,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34255,1676310860648': 2023-02-13 17:55:57,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 17:55:57,000 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 17:55:57,000 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:55:57,000 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:55:57,000 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 17:55:57,000 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,002 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:55:57,002 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:57,003 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,003 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:55:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 17:55:57,005 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,005 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,005 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,011 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 17:55:57,011 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:55:57,011 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:55:57,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:57,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:57,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:55:57,011 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:55:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:55:57,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:55:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:55:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:55:57,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,012 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 17:55:57,112 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:55:57,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:55:57,372 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 17:55:57,373 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:57,374 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,375 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:57,375 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:57,375 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:57,402 ERROR [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 9eb8ad26-3d72-4195-a2ef-90d725f84f89) service to localhost/127.0.0.1:37271 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:57,624 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:57,624 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:57,625 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:57,625 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:57,626 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 17:55:57,627 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:57,627 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:57,627 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:57,627 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:57,627 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,628 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:57,628 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,628 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,628 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,628 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,629 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:57,630 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,631 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,631 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,631 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,631 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:57,631 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,632 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:57,632 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,633 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,633 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,633 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 17:55:57,633 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:57,634 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:57,636 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310987636 2023-02-13 17:55:57,638 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:57,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,641 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:57,641 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:57,641 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:57,641 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311017641 2023-02-13 17:55:57,641 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311077641 2023-02-13 17:55:57,641 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:55:57,641 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:57,643 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:57,643 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:57,643 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:57,891 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,891 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:57,891 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:57,892 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:57,892 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:57,892 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:57,892 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:57,892 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:57,892 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:57,893 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:57,894 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 17:55:57,894 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:57,895 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:57,895 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:57,895 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:57,895 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,896 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:57,897 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,898 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:57,898 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,898 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:57,898 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,899 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,900 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:57,900 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,900 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,901 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,901 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 17:55:57,901 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:57,902 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:57,902 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310987902 2023-02-13 17:55:57,903 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:57,903 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,906 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:57,906 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:57,906 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:57,906 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311017906 2023-02-13 17:55:57,906 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311077906 2023-02-13 17:55:57,906 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:55:57,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:57,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN}] 2023-02-13 17:55:57,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:57,908 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:57,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:57,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:58,156 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,156 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:58,156 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:58,157 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:58,157 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:58,157 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:58,157 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:58,157 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:58,157 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:58,158 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:58,159 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 17:55:58,159 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:58,159 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:58,160 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:58,160 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:58,160 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,160 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:58,161 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,161 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,161 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,161 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:58,162 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,162 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,162 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,162 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:58,163 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:58,163 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,163 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,163 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,163 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,164 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,165 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,165 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,165 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:58,166 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,166 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,166 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,166 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,166 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,167 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:58,167 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,167 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:58,167 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,167 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,167 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,167 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 17:55:58,168 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:58,169 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:58,177 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310988177 2023-02-13 17:55:58,178 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:58,179 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,181 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:58,182 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:58,182 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:58,182 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311018182 2023-02-13 17:55:58,182 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311078182 2023-02-13 17:55:58,182 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:55:58,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN}] 2023-02-13 17:55:58,182 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:58,185 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:55:58,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2ad108a993b4834a8be9722478b2d883, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:58,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310958185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310958185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310958185"}]},"ts":"1676310958185"} 2023-02-13 17:55:58,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:55:58,338 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2ad108a993b4834a8be9722478b2d883, disabling compactions & flushes 2023-02-13 17:55:58,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:58,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:58,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. after waiting 0 ms 2023-02-13 17:55:58,339 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:58,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2ad108a993b4834a8be9722478b2d883 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:55:58,348 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/9c6c20b044f947f0a0ac669a199e5e00 2023-02-13 17:55:58,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/a8307697af044634b12b850334e17023 2023-02-13 17:55:58,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf1/9c6c20b044f947f0a0ac669a199e5e00 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 2023-02-13 17:55:58,365 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:55:58,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/.tmp/cf2/a8307697af044634b12b850334e17023 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 2023-02-13 17:55:58,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:55:58,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 31ms, sequenceid=82, compaction requested=true 2023-02-13 17:55:58,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:55:58,373 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:55:58,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. 2023-02-13 17:55:58,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2ad108a993b4834a8be9722478b2d883: Waiting for close lock at 1676310958339Running coprocessor pre-close hooks at 1676310958339Disabling compacts and flushes for region at 1676310958339Disabling writes for close at 1676310958339Obtaining lock to block concurrent updates at 1676310958339Preparing flush snapshotting stores in 2ad108a993b4834a8be9722478b2d883 at 1676310958339Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310958339Flushing stores of testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. at 1676310958339Flushing 2ad108a993b4834a8be9722478b2d883/cf1: creating writer at 1676310958340 (+1 ms)Flushing 2ad108a993b4834a8be9722478b2d883/cf1: appending metadata at 1676310958342 (+2 ms)Flushing 2ad108a993b4834a8be9722478b2d883/cf1: closing flushed file at 1676310958342Flushing 2ad108a993b4834a8be9722478b2d883/cf2: creating writer at 1676310958352 (+10 ms)Flushing 2ad108a993b4834a8be9722478b2d883/cf2: appending metadata at 1676310958354 (+2 ms)Flushing 2ad108a993b4834a8be9722478b2d883/cf2: closing flushed file at 1676310958354Flushing 2ad108a993b4834a8be9722478b2d883/cf1: reopening flushed file at 1676310958363 (+9 ms)Flushing 2ad108a993b4834a8be9722478b2d883/cf2: reopening flushed file at 1676310958366 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2ad108a993b4834a8be9722478b2d883 in 31ms, sequenceid=82, compaction requested=true at 1676310958370 (+4 ms)Writing region close event to WAL at 1676310958370Running coprocessor post-close hooks at 1676310958373 (+3 ms)Closed at 1676310958374 (+1 ms) 2023-02-13 17:55:58,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED 2023-02-13 17:55:58,377 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310958376"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310958376"}]},"ts":"1676310958376"} 2023-02-13 17:55:58,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 17:55:58,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 in 192 msec 2023-02-13 17:55:58,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 17:55:58,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN in 197 msec 2023-02-13 17:55:58,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:58,383 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:58,383 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:58,432 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,432 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:58,433 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:58,433 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:58,434 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:58,435 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 17:55:58,435 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:55:58,435 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:55:58,435 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:58,436 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:58,436 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:58,436 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:58,436 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,437 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,438 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,438 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:58,438 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:58,439 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,439 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,439 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,439 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:58,439 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,440 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,440 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,440 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:58,440 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,441 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,441 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,441 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,442 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:58,442 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,442 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,442 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,442 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,443 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,443 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:58,443 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,443 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:55:58,443 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:58,444 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:58,444 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,444 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,444 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,444 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:58,444 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:58,444 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 17:55:58,447 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:58,447 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:58,447 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310988447 2023-02-13 17:55:58,448 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:58,449 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,451 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:58,451 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:58,451 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:58,451 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311018451 2023-02-13 17:55:58,452 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311078452 2023-02-13 17:55:58,452 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:55:58,454 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:58,461 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2ad108a993b4834a8be9722478b2d883, threads=14 2023-02-13 17:55:58,462 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,462 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,462 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,462 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,462 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,463 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,463 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,463 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,464 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,464 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,464 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,465 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,468 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,463 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,609 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,609 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,624 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,627 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,628 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,644 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,645 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,649 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,651 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,653 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,654 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,654 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,657 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,663 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,663 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2ad108a993b4834a8be9722478b2d883 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:55:58,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:58,688 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:58,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:58,702 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:58,702 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:58,703 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:58,704 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 17:55:58,704 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:55:58,704 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:55:58,704 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:58,705 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,706 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,707 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,708 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:58,709 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,709 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,709 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:58,709 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:58,709 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:55:58,710 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:58,710 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:58,711 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310988711 2023-02-13 17:55:58,711 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:58,712 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:58,715 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:58,715 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:58,715 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:58,715 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311018715 2023-02-13 17:55:58,715 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311078715 2023-02-13 17:55:58,715 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:55:58,726 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=2ad108a993b4834a8be9722478b2d883, threads=14 2023-02-13 17:55:58,726 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,726 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,726 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,726 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,726 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,730 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,730 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,730 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,730 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,730 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,732 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,732 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,732 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,733 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,801 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,807 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,807 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,808 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,812 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,812 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,817 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,818 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,818 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,820 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,820 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,820 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:58,838 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:55:59,173 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:59,219 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 for region: 2ad108a993b4834a8be9722478b2d883 2023-02-13 17:55:59,219 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 2ad108a993b4834a8be9722478b2d883 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:55:59,249 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:59,254 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:55:59,256 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:55:59,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:59,256 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:59,256 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:59,466 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:59,466 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:59,467 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:59,468 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 17:55:59,468 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:55:59,468 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:55:59,468 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:59,469 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:59,469 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:59,469 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:59,470 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,470 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:59,470 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,470 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,470 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,471 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:59,471 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,471 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,471 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,472 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:59,472 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:59,472 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,472 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,472 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,473 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:59,473 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,473 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,473 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,473 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:59,474 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,474 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,474 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,474 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,474 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:59,475 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,475 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,475 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,475 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,475 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,476 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:59,476 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,476 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:55:59,476 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:59,476 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:59,476 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,477 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,477 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,477 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:59,477 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:59,477 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 17:55:59,478 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:59,478 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:59,478 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310989478 2023-02-13 17:55:59,479 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:59,480 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,482 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:59,482 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:59,482 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:59,482 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311019482 2023-02-13 17:55:59,482 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311079482 2023-02-13 17:55:59,482 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:55:59,485 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:59,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:59,486 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:59,486 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:55:59,733 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:55:59,733 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:55:59,734 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:55:59,735 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 17:55:59,735 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:55:59,735 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:55:59,735 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,736 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,737 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:55:59,738 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,739 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,740 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:55:59,740 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:55:59,740 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,740 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:55:59,740 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:55:59,741 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:55:59,741 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,741 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,741 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,741 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:59,741 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:55:59,741 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 17:55:59,742 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:55:59,742 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:55:59,752 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310989752 2023-02-13 17:55:59,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,754 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:55:59,757 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:55:59,757 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:55:59,757 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:55:59,757 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311019757 2023-02-13 17:55:59,757 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311079757 2023-02-13 17:55:59,757 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:55:59,757 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:55:59,760 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310959760"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676310959760"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676310959760"}]},"ts":"1676310959760"} 2023-02-13 17:55:59,760 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310959760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310959760"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310959760"}]},"ts":"1676310959760"} 2023-02-13 17:55:59,760 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310959760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310959760"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310959760"}]},"ts":"1676310959760"} 2023-02-13 17:55:59,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:55:59,764 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:55:59,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:55:59,765 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:56:00,007 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:56:00,008 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:56:00,008 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:56:00,009 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:56:00,010 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 17:56:00,010 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:56:00,010 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:56:00,010 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:56:00,011 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:56:00,011 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:56:00,011 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:56:00,012 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,012 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:56:00,012 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,012 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,012 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,013 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:56:00,013 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,013 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,013 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:56:00,014 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,015 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,016 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:56:00,017 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,017 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,017 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,017 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,017 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 17:56:00,018 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:56:00,018 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:56:00,019 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310990019 2023-02-13 17:56:00,020 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:56:00,020 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,023 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:56:00,023 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:56:00,023 INFO [Listener at localhost/44157] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648, table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:00,023 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=66330cdcbab53f237a82ae88943c832e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,023 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=b2c06a50cb6d4da2430236423b3b135d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,023 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:56:00,023 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311020023 2023-02-13 17:56:00,023 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311080023 2023-02-13 17:56:00,024 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 17:56:00,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676310960024"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676310960024"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676310960024"}]},"ts":"1676310960024"} 2023-02-13 17:56:00,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310960024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310960024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310960024"}]},"ts":"1676310960024"} 2023-02-13 17:56:00,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310960024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310960024"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310960024"}]},"ts":"1676310960024"} 2023-02-13 17:56:00,026 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:56:00,028 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:56:00,028 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:56:00,028 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:56:00,274 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:56:00,274 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:56:00,275 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:56:00,276 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 17:56:00,276 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:56:00,276 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:56:00,277 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:56:00,277 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:56:00,277 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:56:00,278 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:56:00,278 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,278 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:56:00,278 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,278 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:56:00,279 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,280 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,281 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:56:00,282 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,282 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,282 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,282 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,283 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,283 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 17:56:00,283 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:56:00,284 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:56:00,284 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310990284 2023-02-13 17:56:00,285 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:56:00,285 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,288 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:56:00,288 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:56:00,288 INFO [Listener at localhost/44157] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648, table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:00,288 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=66330cdcbab53f237a82ae88943c832e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,288 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=b2c06a50cb6d4da2430236423b3b135d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,288 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:56:00,288 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311020288 2023-02-13 17:56:00,288 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311080288 2023-02-13 17:56:00,288 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 17:56:00,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:56:00,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN}] 2023-02-13 17:56:00,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:56:00,290 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:56:00,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:56:00,402 ERROR [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 14616398-3dff-4b20-9b76-b1842232aebe) service to localhost/127.0.0.1:37271 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:56:00,539 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:56:00,539 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:56:00,540 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:56:00,541 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 17:56:00,541 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:56:00,541 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:56:00,541 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:56:00,541 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:00,542 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:00,543 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:00,544 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:00,545 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:56:00,546 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,546 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,546 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,546 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:00,546 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 17:56:00,547 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:56:00,547 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:56:00,547 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310990547 2023-02-13 17:56:00,548 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:56:00,549 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,551 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:56:00,551 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:56:00,551 INFO [Listener at localhost/44157] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648, table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:00,551 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=66330cdcbab53f237a82ae88943c832e, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,551 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=b2c06a50cb6d4da2430236423b3b135d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 17:56:00,551 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:56:00,552 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311020552 2023-02-13 17:56:00,552 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311080552 2023-02-13 17:56:00,552 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:56:00,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN}] 2023-02-13 17:56:00,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:56:00,554 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN 2023-02-13 17:56:00,554 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648; forceNewPlan=false, retain=false 2023-02-13 17:56:00,555 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN 2023-02-13 17:56:00,555 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648; forceNewPlan=false, retain=false 2023-02-13 17:56:00,704 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:56:00,706 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b2c06a50cb6d4da2430236423b3b135d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,706 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310960706"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310960706"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310960706"}]},"ts":"1676310960706"} 2023-02-13 17:56:00,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:56:00,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=66330cdcbab53f237a82ae88943c832e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,710 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310960709"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310960709"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310960709"}]},"ts":"1676310960709"} 2023-02-13 17:56:00,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:56:00,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b2c06a50cb6d4da2430236423b3b135d, NAME => 'testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 17:56:00,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:00,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:00,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,863 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,864 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1 2023-02-13 17:56:00,864 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1 2023-02-13 17:56:00,864 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2c06a50cb6d4da2430236423b3b135d columnFamilyName cf1 2023-02-13 17:56:00,875 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-top 2023-02-13 17:56:00,878 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-top 2023-02-13 17:56:00,882 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-top 2023-02-13 17:56:00,885 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-top 2023-02-13 17:56:00,888 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-top 2023-02-13 17:56:00,891 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-top 2023-02-13 17:56:00,894 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-top 2023-02-13 17:56:00,894 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(310): Store=b2c06a50cb6d4da2430236423b3b135d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:00,894 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,894 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2 2023-02-13 17:56:00,895 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2 2023-02-13 17:56:00,895 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2c06a50cb6d4da2430236423b3b135d columnFamilyName cf2 2023-02-13 17:56:00,906 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-top 2023-02-13 17:56:00,909 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-top 2023-02-13 17:56:00,912 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-top 2023-02-13 17:56:00,915 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-top 2023-02-13 17:56:00,918 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-top 2023-02-13 17:56:00,922 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-top 2023-02-13 17:56:00,925 DEBUG [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-top 2023-02-13 17:56:00,925 INFO [StoreOpener-b2c06a50cb6d4da2430236423b3b135d-1] regionserver.HStore(310): Store=b2c06a50cb6d4da2430236423b3b135d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:00,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:00,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:00,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b2c06a50cb6d4da2430236423b3b135d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66131037, jitterRate=-0.014570757746696472}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:00,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b2c06a50cb6d4da2430236423b3b135d: Running coprocessor pre-open hook at 1676310960862Writing region info on filesystem at 1676310960862Initializing all the Stores at 1676310960863 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310960863Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310960863Cleaning up temporary data from old regions at 1676310960927 (+64 ms)Cleaning up detritus from prior splits at 1676310960927Running coprocessor post-open hooks at 1676310960929 (+2 ms)Region opened successfully at 1676310960929 2023-02-13 17:56:00,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d., pid=117, masterSystemTime=1676310960859 2023-02-13 17:56:00,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:00,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:56:00,932 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:00,932 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:00,932 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,932 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1912): b2c06a50cb6d4da2430236423b3b135d/cf1 is initiating minor compaction (all files) 2023-02-13 17:56:00,932 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,932 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1912): b2c06a50cb6d4da2430236423b3b135d/cf2 is initiating minor compaction (all files) 2023-02-13 17:56:00,932 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b2c06a50cb6d4da2430236423b3b135d/cf1 in testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,932 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegion(2258): Starting compaction of b2c06a50cb6d4da2430236423b3b135d/cf2 in testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,933 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/.tmp, totalSize=34.9 K 2023-02-13 17:56:00,933 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/.tmp, totalSize=34.9 K 2023-02-13 17:56:00,933 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310956095 2023-02-13 17:56:00,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:00,933 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310956095 2023-02-13 17:56:00,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:00,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 66330cdcbab53f237a82ae88943c832e, NAME => 'testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 17:56:00,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=b2c06a50cb6d4da2430236423b3b135d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:00,934 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310956258 2023-02-13 17:56:00,934 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310956258 2023-02-13 17:56:00,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:00,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:00,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310960934"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310960934"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310960934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310960934"}]},"ts":"1676310960934"} 2023-02-13 17:56:00,934 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310956423 2023-02-13 17:56:00,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:00,934 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310956423 2023-02-13 17:56:00,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:00,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:00,935 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting 1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310956594 2023-02-13 17:56:00,935 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310956594 2023-02-13 17:56:00,935 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting 715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310956762 2023-02-13 17:56:00,935 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310956762 2023-02-13 17:56:00,936 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310956937 2023-02-13 17:56:00,936 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:00,936 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310956937 2023-02-13 17:56:00,937 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310957114 2023-02-13 17:56:00,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-13 17:56:00,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648 in 228 msec 2023-02-13 17:56:00,937 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1 2023-02-13 17:56:00,937 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1 2023-02-13 17:56:00,937 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310957114 2023-02-13 17:56:00,938 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66330cdcbab53f237a82ae88943c832e columnFamilyName cf1 2023-02-13 17:56:00,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN in 385 msec 2023-02-13 17:56:00,950 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] throttle.PressureAwareThroughputController(145): b2c06a50cb6d4da2430236423b3b135d#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:00,953 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-bottom 2023-02-13 17:56:00,957 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-bottom 2023-02-13 17:56:00,961 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] throttle.PressureAwareThroughputController(145): b2c06a50cb6d4da2430236423b3b135d#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:00,962 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-bottom 2023-02-13 17:56:00,969 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-bottom 2023-02-13 17:56:00,973 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-bottom 2023-02-13 17:56:00,975 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/.tmp/cf2/b377d11876c941a6be1e7884172053bc as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b377d11876c941a6be1e7884172053bc 2023-02-13 17:56:00,978 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/.tmp/cf1/314e8f019f5840f98ab2e8d1e3b65b59 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/314e8f019f5840f98ab2e8d1e3b65b59 2023-02-13 17:56:00,978 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-bottom 2023-02-13 17:56:00,979 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b2c06a50cb6d4da2430236423b3b135d/cf2 of b2c06a50cb6d4da2430236423b3b135d into b377d11876c941a6be1e7884172053bc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:00,980 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for b2c06a50cb6d4da2430236423b3b135d: 2023-02-13 17:56:00,980 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d., storeName=b2c06a50cb6d4da2430236423b3b135d/cf2, priority=9, startTime=1676310960930; duration=0sec 2023-02-13 17:56:00,980 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:00,982 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-bottom 2023-02-13 17:56:00,982 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(310): Store=66330cdcbab53f237a82ae88943c832e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:00,982 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:00,982 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b2c06a50cb6d4da2430236423b3b135d/cf1 of b2c06a50cb6d4da2430236423b3b135d into 314e8f019f5840f98ab2e8d1e3b65b59(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:00,982 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b2c06a50cb6d4da2430236423b3b135d: 2023-02-13 17:56:00,982 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d., storeName=b2c06a50cb6d4da2430236423b3b135d/cf1, priority=9, startTime=1676310960930; duration=0sec 2023-02-13 17:56:00,983 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:00,983 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2 2023-02-13 17:56:00,983 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2 2023-02-13 17:56:00,983 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 66330cdcbab53f237a82ae88943c832e columnFamilyName cf2 2023-02-13 17:56:00,995 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-bottom 2023-02-13 17:56:00,998 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-bottom 2023-02-13 17:56:01,000 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-bottom 2023-02-13 17:56:01,003 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-bottom 2023-02-13 17:56:01,006 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-bottom 2023-02-13 17:56:01,009 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-bottom 2023-02-13 17:56:01,012 DEBUG [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-bottom 2023-02-13 17:56:01,012 INFO [StoreOpener-66330cdcbab53f237a82ae88943c832e-1] regionserver.HStore(310): Store=66330cdcbab53f237a82ae88943c832e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:01,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:01,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 66330cdcbab53f237a82ae88943c832e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73482980, jitterRate=0.09498172998428345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:01,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 66330cdcbab53f237a82ae88943c832e: Running coprocessor pre-open hook at 1676310960935Writing region info on filesystem at 1676310960935Initializing all the Stores at 1676310960935Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310960936 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310960936Cleaning up temporary data from old regions at 1676310961014 (+78 ms)Cleaning up detritus from prior splits at 1676310961015 (+1 ms)Running coprocessor post-open hooks at 1676310961017 (+2 ms)Region opened successfully at 1676310961017 2023-02-13 17:56:01,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e., pid=118, masterSystemTime=1676310960859 2023-02-13 17:56:01,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:01,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:56:01,019 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:01,019 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:01,019 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,019 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1912): 66330cdcbab53f237a82ae88943c832e/cf1 is initiating minor compaction (all files) 2023-02-13 17:56:01,019 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,019 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1912): 66330cdcbab53f237a82ae88943c832e/cf2 is initiating minor compaction (all files) 2023-02-13 17:56:01,019 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 66330cdcbab53f237a82ae88943c832e/cf1 in testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,019 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 66330cdcbab53f237a82ae88943c832e/cf2 in testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,020 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/.tmp, totalSize=34.9 K 2023-02-13 17:56:01,020 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/.tmp, totalSize=34.9 K 2023-02-13 17:56:01,021 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310956095 2023-02-13 17:56:01,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,021 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310956095 2023-02-13 17:56:01,021 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310956258 2023-02-13 17:56:01,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=66330cdcbab53f237a82ae88943c832e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,021 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310956258 2023-02-13 17:56:01,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310961021"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310961021"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310961021"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310961021"}]},"ts":"1676310961021"} 2023-02-13 17:56:01,021 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310956423 2023-02-13 17:56:01,022 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310956423 2023-02-13 17:56:01,022 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310956594 2023-02-13 17:56:01,022 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting 1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310956594 2023-02-13 17:56:01,023 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310956762 2023-02-13 17:56:01,023 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting 715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310956762 2023-02-13 17:56:01,023 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310956937 2023-02-13 17:56:01,023 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310956937 2023-02-13 17:56:01,024 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] compactions.Compactor(207): Compacting 9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310957114 2023-02-13 17:56:01,024 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] compactions.Compactor(207): Compacting a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310957114 2023-02-13 17:56:01,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-02-13 17:56:01,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648 in 313 msec 2023-02-13 17:56:01,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 17:56:01,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN in 473 msec 2023-02-13 17:56:01,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 17:56:01,027 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:56:01,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 17:56:01,036 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] throttle.PressureAwareThroughputController(145): 66330cdcbab53f237a82ae88943c832e#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:01,036 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] throttle.PressureAwareThroughputController(145): 66330cdcbab53f237a82ae88943c832e#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:01,052 INFO [Listener at localhost/44157] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,052 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 17:56:01,052 INFO [Listener at localhost/44157] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:56:01,052 INFO [Listener at localhost/44157] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:56:01,052 ERROR [Listener at localhost/44157] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 17:56:01,053 INFO [Listener at localhost/44157] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 17:56:01,053 INFO [Listener at localhost/44157] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 17:56:01,053 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 17:56:01,053 INFO [Listener at localhost/44157] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 17:56:01,053 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 17:56:01,054 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-13 17:56:01,055 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 17:56:01,056 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 17:56:01,056 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/.tmp/cf1/c5d3913bc277430c97080fad330f4d01 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/c5d3913bc277430c97080fad330f4d01 2023-02-13 17:56:01,056 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 17:56:01,056 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 17:56:01,056 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:01,056 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=b84ccdfb3c511e88b7155232983c9cab, daughterA=cd83e9498f4c7fd91cfbe11625a2fb97, daughterB=0347879560430b134d799b126cf81358 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=eb1733af0943ef056fdc5749cdda62fd, daughterA=357e17d9996b2cb73c13bcd478ff4f03, daughterB=abc074cb651d61f0e94ec5028574f5d0 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 17:56:01,057 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=e0234a68997407cc17c3cb2323d2a02f, daughterA=3f75ed0ecbae6a34d42f633446ce9c87, daughterB=8eafaada8ae1d0fb7136ae4ebf3eaa0e 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=e0234a68997407cc17c3cb2323d2a02f, ASSIGN 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=80781b38cce771d677adbb1da581c54b, daughterA=60d3c4b09a775b33cbe1467e336455bb, daughterB=99f89b3e5ad083d89ee10f03c39f0bb5 2023-02-13 17:56:01,058 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=22d2fa7f26f9d3723b00490163024ece, daughterA=db35248b07d43ec3d6842233230b451a, daughterB=5bd57f0845d56cf0851dc8b161781e6d 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 17:56:01,059 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:01,060 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:01,060 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,39639,1676310860746, table=testRollbackForSplitTableRegionWithReplica, region=879f23954ee5dd3e3e7b467a798bb739 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=f38e847ef463413e4b3d7445cb3aeb9e, daughterA=ead07982923bb14e63a7ec581b2094cd, daughterB=f893645b068165acf88d720cd03cda0c 2023-02-13 17:56:01,060 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:01,060 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 17:56:01,060 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:01,061 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:01,061 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 66330cdcbab53f237a82ae88943c832e/cf1 of 66330cdcbab53f237a82ae88943c832e into c5d3913bc277430c97080fad330f4d01(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:01,061 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 17:56:01,061 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 66330cdcbab53f237a82ae88943c832e: 2023-02-13 17:56:01,061 INFO [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e., storeName=66330cdcbab53f237a82ae88943c832e/cf1, priority=9, startTime=1676310961018; duration=0sec 2023-02-13 17:56:01,061 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,061 DEBUG [RS:0;jenkins-hbase4:34255-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,062 DEBUG [Listener at localhost/44157] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d held the lock before restarting, call acquireLock to restore it. 2023-02-13 17:56:01,062 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,062 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,062 INFO [Listener at localhost/44157] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 2ad108a993b4834a8be9722478b2d883, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 17:56:01,063 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 17:56:01,063 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 17:56:01,064 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699, table=hbase:meta, region=1588230740 2023-02-13 17:56:01,068 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676310991068 2023-02-13 17:56:01,070 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,070 DEBUG [Listener at localhost/44157] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 17:56:01,073 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=4c0f6985b87a7d952cfd36bf98a5ce33, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46577,1676310860699, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699, openSeqNum=2 2023-02-13 17:56:01,073 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=2ad108a993b4834a8be9722478b2d883, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=2 2023-02-13 17:56:01,073 INFO [Listener at localhost/44157] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34255,1676310860648, table=testRecoveryAndDoubleExecution, region=2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,074 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=66330cdcbab53f237a82ae88943c832e, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=86 2023-02-13 17:56:01,074 DEBUG [Listener at localhost/44157] assignment.RegionStateStore(147): Load hbase:meta entry region=b2c06a50cb6d4da2430236423b3b135d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34255,1676310860648, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648, openSeqNum=86 2023-02-13 17:56:01,074 INFO [Listener at localhost/44157] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 17:56:01,074 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676311021074 2023-02-13 17:56:01,074 INFO [Listener at localhost/44157] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676311081074 2023-02-13 17:56:01,074 INFO [Listener at localhost/44157] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 17:56:01,074 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 17:56:01,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=2ad108a993b4834a8be9722478b2d883, daughterA=66330cdcbab53f237a82ae88943c832e, daughterB=b2c06a50cb6d4da2430236423b3b135d in 3.7010 sec 2023-02-13 17:56:01,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34255] ipc.CallRunner(144): callId: 860 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:59768 deadline: 1676311021325, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883. is not online on jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,436 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:56:01,443 INFO [Listener at localhost/44157] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 17:56:01,444 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:56:01,458 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/.tmp/cf2/f3ad6db98f734f449475858d2fef2ee0 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/f3ad6db98f734f449475858d2fef2ee0 2023-02-13 17:56:01,460 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:56:01,462 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 17:56:01,462 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 66330cdcbab53f237a82ae88943c832e/cf2 of 66330cdcbab53f237a82ae88943c832e into f3ad6db98f734f449475858d2fef2ee0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:01,462 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 66330cdcbab53f237a82ae88943c832e: 2023-02-13 17:56:01,462 INFO [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e., storeName=66330cdcbab53f237a82ae88943c832e/cf2, priority=9, startTime=1676310961019; duration=0sec 2023-02-13 17:56:01,462 DEBUG [RS:0;jenkins-hbase4:34255-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:01,462 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-13 17:56:01,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 17:56:01,466 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310961465"}]},"ts":"1676310961465"} 2023-02-13 17:56:01,466 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 17:56:01,468 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 17:56:01,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, UNASSIGN}] 2023-02-13 17:56:01,469 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, UNASSIGN 2023-02-13 17:56:01,469 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b2c06a50cb6d4da2430236423b3b135d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310961469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310961469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310961469"}]},"ts":"1676310961469"} 2023-02-13 17:56:01,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:56:01,472 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, UNASSIGN 2023-02-13 17:56:01,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=66330cdcbab53f237a82ae88943c832e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:01,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310961472"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310961472"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310961472"}]},"ts":"1676310961472"} 2023-02-13 17:56:01,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648}] 2023-02-13 17:56:01,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 17:56:01,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b2c06a50cb6d4da2430236423b3b135d, disabling compactions & flushes 2023-02-13 17:56:01,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:01,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:01,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. after waiting 0 ms 2023-02-13 17:56:01,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:01,623 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-top] to archive 2023-02-13 17:56:01,624 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:01,625 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,626 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,627 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,628 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,629 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,630 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,631 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,632 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-top] to archive 2023-02-13 17:56:01,633 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:01,634 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,635 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,636 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,637 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,638 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,638 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,639 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,642 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:56:01,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:01,643 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d. 2023-02-13 17:56:01,643 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b2c06a50cb6d4da2430236423b3b135d: Waiting for close lock at 1676310961622Running coprocessor pre-close hooks at 1676310961622Disabling compacts and flushes for region at 1676310961622Disabling writes for close at 1676310961622Writing region close event to WAL at 1676310961640 (+18 ms)Running coprocessor post-close hooks at 1676310961643 (+3 ms)Closed at 1676310961643 2023-02-13 17:56:01,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 66330cdcbab53f237a82ae88943c832e, disabling compactions & flushes 2023-02-13 17:56:01,645 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. after waiting 0 ms 2023-02-13 17:56:01,645 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,645 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=b2c06a50cb6d4da2430236423b3b135d, regionState=CLOSED 2023-02-13 17:56:01,645 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310961645"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310961645"}]},"ts":"1676310961645"} 2023-02-13 17:56:01,646 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00-bottom] to archive 2023-02-13 17:56:01,647 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:01,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 17:56:01,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure b2c06a50cb6d4da2430236423b3b135d, server=jenkins-hbase4.apache.org,34255,1676310860648 in 177 msec 2023-02-13 17:56:01,649 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=b2c06a50cb6d4da2430236423b3b135d, UNASSIGN in 180 msec 2023-02-13 17:56:01,650 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,651 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,651 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,652 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,653 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,654 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,655 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023-bottom] to archive 2023-02-13 17:56:01,656 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:01,657 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,658 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,659 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,659 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,660 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,661 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,662 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:56:01,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:01,666 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e. 2023-02-13 17:56:01,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 66330cdcbab53f237a82ae88943c832e: Waiting for close lock at 1676310961645Running coprocessor pre-close hooks at 1676310961645Disabling compacts and flushes for region at 1676310961645Disabling writes for close at 1676310961645Writing region close event to WAL at 1676310961663 (+18 ms)Running coprocessor post-close hooks at 1676310961666 (+3 ms)Closed at 1676310961666 2023-02-13 17:56:01,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,671 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=66330cdcbab53f237a82ae88943c832e, regionState=CLOSED 2023-02-13 17:56:01,672 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676310961671"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310961671"}]},"ts":"1676310961671"} 2023-02-13 17:56:01,674 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 17:56:01,674 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 66330cdcbab53f237a82ae88943c832e, server=jenkins-hbase4.apache.org,34255,1676310860648 in 200 msec 2023-02-13 17:56:01,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-13 17:56:01,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=66330cdcbab53f237a82ae88943c832e, UNASSIGN in 206 msec 2023-02-13 17:56:01,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310961677"}]},"ts":"1676310961677"} 2023-02-13 17:56:01,678 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 17:56:01,680 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 17:56:01,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 217 msec 2023-02-13 17:56:01,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 17:56:01,767 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 17:56:01,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-13 17:56:01,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 17:56:01,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,775 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,775 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,775 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,777 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/recovered.edits] 2023-02-13 17:56:01,777 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/recovered.edits] 2023-02-13 17:56:01,777 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/recovered.edits] 2023-02-13 17:56:01,780 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/.regioninfo to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/.regioninfo 2023-02-13 17:56:01,780 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/.regioninfo to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/.regioninfo 2023-02-13 17:56:01,780 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2550fec6a72945508c0ca3bcdaa720c9 2023-02-13 17:56:01,782 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/2d18a6a534f140be834f4ca3eb4beae6 2023-02-13 17:56:01,783 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,783 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/3c921158225d4063b857a4f3036e7353 2023-02-13 17:56:01,784 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/2550fec6a72945508c0ca3bcdaa720c9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,784 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,785 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/58592fd9d025457a8817194dea55c357 2023-02-13 17:56:01,785 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/2d18a6a534f140be834f4ca3eb4beae6.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,786 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,786 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/78cf0b64627a457b8e24bdc678e9c369 2023-02-13 17:56:01,787 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/3c921158225d4063b857a4f3036e7353.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,788 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,788 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/99160cfab74648399fdddbbea98695ac 2023-02-13 17:56:01,788 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/58592fd9d025457a8817194dea55c357.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,789 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,789 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf1/9c6c20b044f947f0a0ac669a199e5e00 2023-02-13 17:56:01,790 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/78cf0b64627a457b8e24bdc678e9c369.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,790 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,791 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/99160cfab74648399fdddbbea98695ac.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,792 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,792 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3 2023-02-13 17:56:01,793 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf1/9c6c20b044f947f0a0ac669a199e5e00.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,794 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/715f98e6f39945f6b965f895e41819a9 2023-02-13 17:56:01,795 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,796 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/a8307697af044634b12b850334e17023 2023-02-13 17:56:01,797 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,797 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/1a82d5b5d77b4e7c9ecfbcb4edd450e3.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,798 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/aba94b15b2fe446f8b8e53cdc90776e2 2023-02-13 17:56:01,798 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,799 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/715f98e6f39945f6b965f895e41819a9.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,799 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b882c9709fbe446d96d5d9abdca229a7 2023-02-13 17:56:01,800 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,800 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/a8307697af044634b12b850334e17023.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,801 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c 2023-02-13 17:56:01,801 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,802 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/aba94b15b2fe446f8b8e53cdc90776e2.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,802 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/cf2/e83a1858084c45e08fb24dc03f3d540d 2023-02-13 17:56:01,803 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,804 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/b882c9709fbe446d96d5d9abdca229a7.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,804 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/66330cdcbab53f237a82ae88943c832e/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,805 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/b8b2f2d7882b4ba8809f1cc0d7b1ed3c.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,806 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883/recovered.edits/85.seqid 2023-02-13 17:56:01,806 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,806 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/b2c06a50cb6d4da2430236423b3b135d/cf2/e83a1858084c45e08fb24dc03f3d540d.2ad108a993b4834a8be9722478b2d883 2023-02-13 17:56:01,807 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/c5d3913bc277430c97080fad330f4d01 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf1/c5d3913bc277430c97080fad330f4d01 2023-02-13 17:56:01,808 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/314e8f019f5840f98ab2e8d1e3b65b59 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf1/314e8f019f5840f98ab2e8d1e3b65b59 2023-02-13 17:56:01,808 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/f3ad6db98f734f449475858d2fef2ee0 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/cf2/f3ad6db98f734f449475858d2fef2ee0 2023-02-13 17:56:01,810 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b377d11876c941a6be1e7884172053bc to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/cf2/b377d11876c941a6be1e7884172053bc 2023-02-13 17:56:01,811 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e/recovered.edits/90.seqid 2023-02-13 17:56:01,811 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/66330cdcbab53f237a82ae88943c832e 2023-02-13 17:56:01,812 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d/recovered.edits/90.seqid 2023-02-13 17:56:01,813 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testRecoveryAndDoubleExecution/b2c06a50cb6d4da2430236423b3b135d 2023-02-13 17:56:01,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 17:56:01,815 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,816 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 17:56:01,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 17:56:01,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,818 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 17:56:01,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310961818"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:01,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310961818"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:01,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310961818"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:01,820 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:56:01,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2ad108a993b4834a8be9722478b2d883, NAME => 'testRecoveryAndDoubleExecution,,1676310955587.2ad108a993b4834a8be9722478b2d883.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 66330cdcbab53f237a82ae88943c832e, NAME => 'testRecoveryAndDoubleExecution,,1676310957373.66330cdcbab53f237a82ae88943c832e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b2c06a50cb6d4da2430236423b3b135d, NAME => 'testRecoveryAndDoubleExecution,41,1676310957373.b2c06a50cb6d4da2430236423b3b135d.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 17:56:01,820 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 17:56:01,820 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310961820"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:01,821 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 17:56:01,822 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 17:56:01,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 55 msec 2023-02-13 17:56:01,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 17:56:01,872 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 17:56:01,888 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=376 (was 361) - Thread LEAK? -, OpenFileDescriptor=844 (was 736) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 138), ProcessCount=168 (was 168), AvailableMemoryMB=858 (was 911) 2023-02-13 17:56:01,900 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=168, AvailableMemoryMB=858 2023-02-13 17:56:01,901 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:56:01,904 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:56:01,905 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:56:01,906 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:01,907 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:56:01,907 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:56:01,909 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:01,909 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 empty. 2023-02-13 17:56:01,909 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:01,909 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 17:56:01,918 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 17:56:01,919 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 09e0c5ec0772f8303a6c93b7fa1dfc73, NAME => 'testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 09e0c5ec0772f8303a6c93b7fa1dfc73, disabling compactions & flushes 2023-02-13 17:56:01,925 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. after waiting 0 ms 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:01,925 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:01,925 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: Waiting for close lock at 1676310961925Disabling compacts and flushes for region at 1676310961925Disabling writes for close at 1676310961925Writing region close event to WAL at 1676310961925Closed at 1676310961925 2023-02-13 17:56:01,926 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:56:01,927 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310961927"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310961927"}]},"ts":"1676310961927"} 2023-02-13 17:56:01,928 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:56:01,929 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:56:01,929 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310961929"}]},"ts":"1676310961929"} 2023-02-13 17:56:01,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 17:56:01,932 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:56:01,933 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:56:01,933 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:56:01,933 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:56:01,933 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:56:01,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, ASSIGN}] 2023-02-13 17:56:01,934 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, ASSIGN 2023-02-13 17:56:01,934 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:56:02,085 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:56:02,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=09e0c5ec0772f8303a6c93b7fa1dfc73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310962086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310962086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310962086"}]},"ts":"1676310962086"} 2023-02-13 17:56:02,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 09e0c5ec0772f8303a6c93b7fa1dfc73, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:02,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 09e0c5ec0772f8303a6c93b7fa1dfc73, NAME => 'testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:56:02,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:02,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:02,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,243 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,244 DEBUG [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1 2023-02-13 17:56:02,244 DEBUG [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1 2023-02-13 17:56:02,244 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09e0c5ec0772f8303a6c93b7fa1dfc73 columnFamilyName cf1 2023-02-13 17:56:02,245 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] regionserver.HStore(310): Store=09e0c5ec0772f8303a6c93b7fa1dfc73/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:02,245 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,246 DEBUG [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2 2023-02-13 17:56:02,246 DEBUG [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2 2023-02-13 17:56:02,246 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 09e0c5ec0772f8303a6c93b7fa1dfc73 columnFamilyName cf2 2023-02-13 17:56:02,246 INFO [StoreOpener-09e0c5ec0772f8303a6c93b7fa1dfc73-1] regionserver.HStore(310): Store=09e0c5ec0772f8303a6c93b7fa1dfc73/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:02,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:02,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:02,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:56:02,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 09e0c5ec0772f8303a6c93b7fa1dfc73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75021544, jitterRate=0.11790812015533447}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:02,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: Running coprocessor pre-open hook at 1676310962242Writing region info on filesystem at 1676310962242Initializing all the Stores at 1676310962243 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310962243Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310962243Cleaning up temporary data from old regions at 1676310962248 (+5 ms)Cleaning up detritus from prior splits at 1676310962248Running coprocessor post-open hooks at 1676310962251 (+3 ms)Region opened successfully at 1676310962251 2023-02-13 17:56:02,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73., pid=127, masterSystemTime=1676310962239 2023-02-13 17:56:02,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,253 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=09e0c5ec0772f8303a6c93b7fa1dfc73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,254 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310962253"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310962253"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310962253"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310962253"}]},"ts":"1676310962253"} 2023-02-13 17:56:02,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 17:56:02,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 09e0c5ec0772f8303a6c93b7fa1dfc73, server=jenkins-hbase4.apache.org,46577,1676310860699 in 168 msec 2023-02-13 17:56:02,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 17:56:02,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, ASSIGN in 323 msec 2023-02-13 17:56:02,258 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:56:02,258 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310962258"}]},"ts":"1676310962258"} 2023-02-13 17:56:02,259 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 17:56:02,260 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:56:02,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 355 msec 2023-02-13 17:56:02,413 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:02,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,415 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,415 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,415 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:02,415 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:02,415 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,415 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,417 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,417 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,417 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,417 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,418 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,418 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,419 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,419 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,420 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,420 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,420 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,423 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,423 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,429 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/95e03b821ad4468392186391879f2cef 2023-02-13 17:56:02,439 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/e378335f3898494994e57ddb302e371d 2023-02-13 17:56:02,442 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/95e03b821ad4468392186391879f2cef as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef 2023-02-13 17:56:02,445 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:56:02,445 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/e378335f3898494994e57ddb302e371d as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d 2023-02-13 17:56:02,448 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:56:02,448 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 28ms, sequenceid=6, compaction requested=false 2023-02-13 17:56:02,448 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,449 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:02,449 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,449 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:02,449 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:02,449 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,449 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,449 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,450 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,450 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,451 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:02,452 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1dc840e3[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:02,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:02,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:02,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,455 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,455 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:02,456 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,456 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,456 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,457 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,457 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,458 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:02,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:02,460 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:02,460 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:02,460 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:02,460 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,460 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,463 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,463 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,464 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,464 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,466 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,466 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,466 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,471 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,472 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,471 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,472 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:02,471 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,472 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,472 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,472 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,473 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,572 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:02,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:02,584 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:02,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:02,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,585 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,585 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,585 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:02,585 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:02,586 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,586 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,587 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,587 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,587 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,587 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,589 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,589 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,589 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,589 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,589 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,589 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,590 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,592 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,602 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/bc9b1b11d8234502a5c129cb9cada938 2023-02-13 17:56:02,611 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/2b30f3b0965f4cff8acb4440574229fd 2023-02-13 17:56:02,614 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/bc9b1b11d8234502a5c129cb9cada938 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938 2023-02-13 17:56:02,617 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:56:02,618 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/2b30f3b0965f4cff8acb4440574229fd as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd 2023-02-13 17:56:02,621 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:56:02,622 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 33ms, sequenceid=19, compaction requested=false 2023-02-13 17:56:02,622 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:02,622 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:02,622 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,623 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,624 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,624 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:02,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f39c814[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:02,626 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:02,626 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,627 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,628 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,628 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,628 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:02,630 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,630 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,630 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,631 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,631 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:02,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:02,634 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:02,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:02,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:02,634 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,634 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:02,635 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,636 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,636 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,637 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,637 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,638 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,639 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,639 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,641 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,641 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,641 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,641 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,642 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,742 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:02,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:02,752 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:02,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:02,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,753 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,753 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,753 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:02,753 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:02,754 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,754 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,756 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,756 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,756 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,756 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:02,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,758 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,758 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,758 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,758 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,758 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,758 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:02,758 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,758 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,758 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:02,759 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:02,759 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,759 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,759 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,760 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,760 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,760 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,769 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/fb49e4110f5a418f8a8822cddf15016a 2023-02-13 17:56:02,778 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/31f0d99827134638961bfe1c159a6d15 2023-02-13 17:56:02,781 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/fb49e4110f5a418f8a8822cddf15016a as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a 2023-02-13 17:56:02,784 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:56:02,785 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/31f0d99827134638961bfe1c159a6d15 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15 2023-02-13 17:56:02,787 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:56:02,788 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 29ms, sequenceid=32, compaction requested=true 2023-02-13 17:56:02,788 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:02,788 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,788 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:02,788 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:02,789 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,789 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,789 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,790 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,790 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,790 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:02,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b26f0a4[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:02,792 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:02,792 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,794 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,794 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,795 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:02,794 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,794 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,795 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:02,794 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,795 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:02,795 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,795 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,795 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:02,798 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,798 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,798 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,798 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,799 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:02,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:02,802 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:02,802 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:02,802 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:02,802 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,802 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,804 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,805 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,805 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,806 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,807 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,807 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,809 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,809 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,810 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:02,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,809 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:02,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,810 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,910 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:02,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:02,921 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:02,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:02,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,922 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,922 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,922 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:02,922 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:02,923 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,923 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,924 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,924 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,924 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,924 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:02,926 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,926 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:02,926 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,926 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:02,927 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,927 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,928 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,934 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/b94e913d45a74de5857915fdbd7e2057 2023-02-13 17:56:02,943 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/fc1505727ff64852a648f07b865ee762 2023-02-13 17:56:02,946 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/b94e913d45a74de5857915fdbd7e2057 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057 2023-02-13 17:56:02,949 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:56:02,950 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/fc1505727ff64852a648f07b865ee762 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762 2023-02-13 17:56:02,953 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:56:02,953 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 27ms, sequenceid=45, compaction requested=true 2023-02-13 17:56:02,953 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:02,953 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:02,954 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:02,954 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:02,954 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,954 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:02,954 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:02,956 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,956 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,956 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,956 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:02,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79f100e3[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:02,958 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:02,958 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,959 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:02,959 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:02,960 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:02,961 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,961 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:02,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:02,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:02,964 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:02,964 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:02,964 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:02,964 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,964 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:02,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:02,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,966 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,967 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:02,967 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,969 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:02,969 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,969 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,978 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:02,978 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:02,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,978 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:02,979 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,079 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:03,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:03,090 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:03,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:03,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,092 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,092 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,093 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,093 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,094 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,094 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:03,095 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:03,095 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,096 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:03,096 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:03,096 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,098 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,103 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/701506c46c10402890874c5eff3fccb1 2023-02-13 17:56:03,113 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/b149248029dc44d3b8f90620875ee00f 2023-02-13 17:56:03,116 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/701506c46c10402890874c5eff3fccb1 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1 2023-02-13 17:56:03,119 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:56:03,119 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/b149248029dc44d3b8f90620875ee00f as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f 2023-02-13 17:56:03,122 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:56:03,123 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 27ms, sequenceid=58, compaction requested=true 2023-02-13 17:56:03,123 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:03,123 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,123 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:03,123 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:03,123 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,123 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,123 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,124 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,124 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,125 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:03,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@26e18866[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:03,126 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:03,126 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:03,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,131 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,131 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:03,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:03,134 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:03,134 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:03,134 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:03,134 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,134 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,137 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,137 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,139 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,139 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,139 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,141 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:03,141 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,141 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:03,141 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,142 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,242 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:03,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:03,252 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:03,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:03,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,253 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:03,253 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,253 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 17:56:03,253 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 17:56:03,253 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,254 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,255 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,255 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,255 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,256 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 17:56:03,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,256 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,257 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:03,257 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. started... 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,257 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:03,257 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,260 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,265 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/52bffd3b83574722b30e08ca75fd5da2 2023-02-13 17:56:03,274 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/faf2516095d94943868aaee8ae96622c 2023-02-13 17:56:03,277 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/52bffd3b83574722b30e08ca75fd5da2 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2 2023-02-13 17:56:03,280 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:56:03,281 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/faf2516095d94943868aaee8ae96622c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c 2023-02-13 17:56:03,284 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:56:03,285 INFO [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 28ms, sequenceid=71, compaction requested=true 2023-02-13 17:56:03,285 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: 2023-02-13 17:56:03,285 DEBUG [rs(jenkins-hbase4.apache.org,46577,1676310860699)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 17:56:03,285 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 17:56:03,287 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,287 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,287 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,287 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 17:56:03,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47f99dc6[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:03,289 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 17:56:03,289 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:03,292 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,293 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:03,294 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,294 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,295 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 17:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46577,1676310860699': 2023-02-13 17:56:03,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 17:56:03,297 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 17:56:03,297 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:03,297 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:03,297 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,297 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,299 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:03,300 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,300 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,301 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,302 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:03,302 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,302 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,305 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:03,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 17:56:03,305 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:03,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,305 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:03,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,306 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 17:56:03,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:03,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:03,406 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:03,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:03,413 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,414 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=09e0c5ec0772f8303a6c93b7fa1dfc73, daughterA=56b48036033268a00528783adbdab412, daughterB=cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:03,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=09e0c5ec0772f8303a6c93b7fa1dfc73, daughterA=56b48036033268a00528783adbdab412, daughterB=cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:03,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=09e0c5ec0772f8303a6c93b7fa1dfc73, daughterA=56b48036033268a00528783adbdab412, daughterB=cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:03,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=09e0c5ec0772f8303a6c93b7fa1dfc73, daughterA=56b48036033268a00528783adbdab412, daughterB=cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:03,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, UNASSIGN}] 2023-02-13 17:56:03,417 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, UNASSIGN 2023-02-13 17:56:03,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=09e0c5ec0772f8303a6c93b7fa1dfc73, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,418 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310963417"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310963417"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310963417"}]},"ts":"1676310963417"} 2023-02-13 17:56:03,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 09e0c5ec0772f8303a6c93b7fa1dfc73, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:03,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 09e0c5ec0772f8303a6c93b7fa1dfc73, disabling compactions & flushes 2023-02-13 17:56:03,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. after waiting 0 ms 2023-02-13 17:56:03,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,569 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:56:03,580 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/2e825bbd99484113893a212569a8e0d8 2023-02-13 17:56:03,589 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e 2023-02-13 17:56:03,593 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf1/2e825bbd99484113893a212569a8e0d8 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8 2023-02-13 17:56:03,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:56:03,597 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.tmp/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e 2023-02-13 17:56:03,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:56:03,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 31ms, sequenceid=82, compaction requested=true 2023-02-13 17:56:03,604 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:56:03,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:03,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. 2023-02-13 17:56:03,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 09e0c5ec0772f8303a6c93b7fa1dfc73: Waiting for close lock at 1676310963569Running coprocessor pre-close hooks at 1676310963569Disabling compacts and flushes for region at 1676310963569Disabling writes for close at 1676310963569Obtaining lock to block concurrent updates at 1676310963570 (+1 ms)Preparing flush snapshotting stores in 09e0c5ec0772f8303a6c93b7fa1dfc73 at 1676310963570Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310963570Flushing stores of testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. at 1676310963570Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf1: creating writer at 1676310963573 (+3 ms)Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf1: appending metadata at 1676310963574 (+1 ms)Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf1: closing flushed file at 1676310963574Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf2: creating writer at 1676310963583 (+9 ms)Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf2: appending metadata at 1676310963584 (+1 ms)Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf2: closing flushed file at 1676310963584Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf1: reopening flushed file at 1676310963594 (+10 ms)Flushing 09e0c5ec0772f8303a6c93b7fa1dfc73/cf2: reopening flushed file at 1676310963597 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 09e0c5ec0772f8303a6c93b7fa1dfc73 in 31ms, sequenceid=82, compaction requested=true at 1676310963600 (+3 ms)Writing region close event to WAL at 1676310963602 (+2 ms)Running coprocessor post-close hooks at 1676310963605 (+3 ms)Closed at 1676310963605 2023-02-13 17:56:03,606 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=09e0c5ec0772f8303a6c93b7fa1dfc73, regionState=CLOSED 2023-02-13 17:56:03,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310963606"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310963606"}]},"ts":"1676310963606"} 2023-02-13 17:56:03,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 17:56:03,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 09e0c5ec0772f8303a6c93b7fa1dfc73, server=jenkins-hbase4.apache.org,46577,1676310860699 in 189 msec 2023-02-13 17:56:03,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 17:56:03,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=09e0c5ec0772f8303a6c93b7fa1dfc73, UNASSIGN in 193 msec 2023-02-13 17:56:03,620 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=09e0c5ec0772f8303a6c93b7fa1dfc73, threads=14 2023-02-13 17:56:03,620 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,620 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,624 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,624 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,624 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,624 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,628 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,629 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,629 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,630 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,632 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,632 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,632 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,632 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,634 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd, top=false 2023-02-13 17:56:03,635 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e, top=false 2023-02-13 17:56:03,635 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f, top=false 2023-02-13 17:56:03,636 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c, top=false 2023-02-13 17:56:03,637 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762, top=false 2023-02-13 17:56:03,639 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1, top=false 2023-02-13 17:56:03,639 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a, top=false 2023-02-13 17:56:03,640 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15, top=false 2023-02-13 17:56:03,642 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2, top=false 2023-02-13 17:56:03,643 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef, top=false 2023-02-13 17:56:03,644 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d, top=false 2023-02-13 17:56:03,645 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938, top=false 2023-02-13 17:56:03,646 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057, top=false 2023-02-13 17:56:03,648 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8, top=false 2023-02-13 17:56:03,661 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,661 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,667 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,667 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,668 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,669 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,669 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,671 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,672 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,672 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/.splits/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057 for child: 56b48036033268a00528783adbdab412, parent: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,672 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,672 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,675 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,675 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,678 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,678 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,679 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,679 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,679 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,679 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,680 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,682 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057 for region: 09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:03,682 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 09e0c5ec0772f8303a6c93b7fa1dfc73 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 17:56:03,708 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:56:03,709 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:56:03,710 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676310963710"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676310963710"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676310963710"}]},"ts":"1676310963710"} 2023-02-13 17:56:03,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310963710"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310963710"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310963710"}]},"ts":"1676310963710"} 2023-02-13 17:56:03,711 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310963710"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310963710"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310963710"}]},"ts":"1676310963710"} 2023-02-13 17:56:03,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, ASSIGN}] 2023-02-13 17:56:03,714 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, ASSIGN 2023-02-13 17:56:03,715 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:56:03,715 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, ASSIGN 2023-02-13 17:56:03,716 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46577,1676310860699; forceNewPlan=false, retain=false 2023-02-13 17:56:03,865 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:56:03,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=cae400a17fe2777e2ee20b2518d3e960, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310963867"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310963867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310963867"}]},"ts":"1676310963867"} 2023-02-13 17:56:03,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure cae400a17fe2777e2ee20b2518d3e960, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:03,870 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=56b48036033268a00528783adbdab412, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:03,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310963870"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310963870"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310963870"}]},"ts":"1676310963870"} 2023-02-13 17:56:03,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 56b48036033268a00528783adbdab412, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:03,875 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:56:04,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 56b48036033268a00528783adbdab412, NAME => 'testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 17:56:04,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:04,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:04,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,023 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,023 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1 2023-02-13 17:56:04,023 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1 2023-02-13 17:56:04,024 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56b48036033268a00528783adbdab412 columnFamilyName cf1 2023-02-13 17:56:04,028 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8 2023-02-13 17:56:04,031 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2 2023-02-13 17:56:04,033 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1 2023-02-13 17:56:04,036 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef 2023-02-13 17:56:04,038 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057 2023-02-13 17:56:04,040 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938 2023-02-13 17:56:04,043 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a 2023-02-13 17:56:04,043 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(310): Store=56b48036033268a00528783adbdab412/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:04,043 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,044 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2 2023-02-13 17:56:04,044 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2 2023-02-13 17:56:04,044 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 56b48036033268a00528783adbdab412 columnFamilyName cf2 2023-02-13 17:56:04,048 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e 2023-02-13 17:56:04,050 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd 2023-02-13 17:56:04,052 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15 2023-02-13 17:56:04,055 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f 2023-02-13 17:56:04,057 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d 2023-02-13 17:56:04,060 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c 2023-02-13 17:56:04,063 DEBUG [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762 2023-02-13 17:56:04,063 INFO [StoreOpener-56b48036033268a00528783adbdab412-1] regionserver.HStore(310): Store=56b48036033268a00528783adbdab412/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:04,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412 2023-02-13 17:56:04,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412 2023-02-13 17:56:04,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:04,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 56b48036033268a00528783adbdab412; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74682388, jitterRate=0.11285430192947388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:04,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 56b48036033268a00528783adbdab412: Running coprocessor pre-open hook at 1676310964022Writing region info on filesystem at 1676310964022Initializing all the Stores at 1676310964022Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310964022Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310964022Cleaning up temporary data from old regions at 1676310964064 (+42 ms)Cleaning up detritus from prior splits at 1676310964065 (+1 ms)Running coprocessor post-open hooks at 1676310964066 (+1 ms)Region opened successfully at 1676310964066 2023-02-13 17:56:04,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412., pid=134, masterSystemTime=1676310964019 2023-02-13 17:56:04,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:04,068 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1912): 56b48036033268a00528783adbdab412/cf1 is initiating minor compaction (all files) 2023-02-13 17:56:04,068 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1912): 56b48036033268a00528783adbdab412/cf2 is initiating minor compaction (all files) 2023-02-13 17:56:04,068 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 56b48036033268a00528783adbdab412/cf1 in testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,068 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 56b48036033268a00528783adbdab412/cf2 in testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,069 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/.tmp, totalSize=34.9 K 2023-02-13 17:56:04,069 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/.tmp, totalSize=34.9 K 2023-02-13 17:56:04,069 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310962408 2023-02-13 17:56:04,069 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310962408 2023-02-13 17:56:04,069 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,069 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310962574 2023-02-13 17:56:04,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,069 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310962574 2023-02-13 17:56:04,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cae400a17fe2777e2ee20b2518d3e960, NAME => 'testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 17:56:04,070 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=56b48036033268a00528783adbdab412, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:04,070 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310962743 2023-02-13 17:56:04,070 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964070"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310964070"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310964070"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310964070"}]},"ts":"1676310964070"} 2023-02-13 17:56:04,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:04,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,070 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310962743 2023-02-13 17:56:04,070 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310962912 2023-02-13 17:56:04,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:04,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,071 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,071 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310962912 2023-02-13 17:56:04,071 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310963081 2023-02-13 17:56:04,071 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310963081 2023-02-13 17:56:04,072 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310963243 2023-02-13 17:56:04,072 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,072 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310963243 2023-02-13 17:56:04,072 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310963407 2023-02-13 17:56:04,072 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310963407 2023-02-13 17:56:04,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-13 17:56:04,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 56b48036033268a00528783adbdab412, server=jenkins-hbase4.apache.org,46577,1676310860699 in 199 msec 2023-02-13 17:56:04,075 DEBUG [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf1 2023-02-13 17:56:04,075 DEBUG [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf1 2023-02-13 17:56:04,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, ASSIGN in 361 msec 2023-02-13 17:56:04,076 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cae400a17fe2777e2ee20b2518d3e960 columnFamilyName cf1 2023-02-13 17:56:04,076 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] regionserver.HStore(310): Store=cae400a17fe2777e2ee20b2518d3e960/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:04,076 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,078 DEBUG [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf2 2023-02-13 17:56:04,078 DEBUG [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf2 2023-02-13 17:56:04,079 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cae400a17fe2777e2ee20b2518d3e960 columnFamilyName cf2 2023-02-13 17:56:04,079 INFO [StoreOpener-cae400a17fe2777e2ee20b2518d3e960-1] regionserver.HStore(310): Store=cae400a17fe2777e2ee20b2518d3e960/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:04,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:04,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cae400a17fe2777e2ee20b2518d3e960; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71675325, jitterRate=0.06804557144641876}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:04,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cae400a17fe2777e2ee20b2518d3e960: Running coprocessor pre-open hook at 1676310964071Writing region info on filesystem at 1676310964071Initializing all the Stores at 1676310964072 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310964072Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310964072Cleaning up temporary data from old regions at 1676310964082 (+10 ms)Cleaning up detritus from prior splits at 1676310964082Running coprocessor post-open hooks at 1676310964085 (+3 ms)Region opened successfully at 1676310964085 2023-02-13 17:56:04,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960., pid=133, masterSystemTime=1676310964019 2023-02-13 17:56:04,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=cae400a17fe2777e2ee20b2518d3e960, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:04,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964087"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310964087"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310964087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310964087"}]},"ts":"1676310964087"} 2023-02-13 17:56:04,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-13 17:56:04,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure cae400a17fe2777e2ee20b2518d3e960, server=jenkins-hbase4.apache.org,46577,1676310860699 in 221 msec 2023-02-13 17:56:04,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 17:56:04,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, ASSIGN in 377 msec 2023-02-13 17:56:04,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=09e0c5ec0772f8303a6c93b7fa1dfc73, daughterA=56b48036033268a00528783adbdab412, daughterB=cae400a17fe2777e2ee20b2518d3e960 in 678 msec 2023-02-13 17:56:04,099 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] throttle.PressureAwareThroughputController(145): 56b48036033268a00528783adbdab412#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:04,099 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] throttle.PressureAwareThroughputController(145): 56b48036033268a00528783adbdab412#cf2#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:04,120 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/.tmp/cf1/c2af6732de224d42a6a62e58f66d9fcb as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/c2af6732de224d42a6a62e58f66d9fcb 2023-02-13 17:56:04,122 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/.tmp/cf2/4538a88d70fa4887bad66792b52e56ae as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/4538a88d70fa4887bad66792b52e56ae 2023-02-13 17:56:04,124 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 56b48036033268a00528783adbdab412/cf1 of 56b48036033268a00528783adbdab412 into c2af6732de224d42a6a62e58f66d9fcb(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:04,124 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 56b48036033268a00528783adbdab412: 2023-02-13 17:56:04,124 INFO [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412., storeName=56b48036033268a00528783adbdab412/cf1, priority=9, startTime=1676310964067; duration=0sec 2023-02-13 17:56:04,124 DEBUG [RS:1;jenkins-hbase4:46577-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:04,126 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 56b48036033268a00528783adbdab412/cf2 of 56b48036033268a00528783adbdab412 into 4538a88d70fa4887bad66792b52e56ae(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:04,126 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 56b48036033268a00528783adbdab412: 2023-02-13 17:56:04,127 INFO [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412., storeName=56b48036033268a00528783adbdab412/cf2, priority=9, startTime=1676310964068; duration=0sec 2023-02-13 17:56:04,127 DEBUG [RS:1;jenkins-hbase4:46577-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:04,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46577] ipc.CallRunner(144): callId: 991 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40778 deadline: 1676311024164, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73. is not online on jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:04,274 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 17:56:04,277 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:56:04,279 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,279 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 17:56:04,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310964282"}]},"ts":"1676310964282"} 2023-02-13 17:56:04,283 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 17:56:04,284 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 17:56:04,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, UNASSIGN}] 2023-02-13 17:56:04,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, UNASSIGN 2023-02-13 17:56:04,286 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=cae400a17fe2777e2ee20b2518d3e960, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:04,286 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964286"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310964286"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310964286"}]},"ts":"1676310964286"} 2023-02-13 17:56:04,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure cae400a17fe2777e2ee20b2518d3e960, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:04,289 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, UNASSIGN 2023-02-13 17:56:04,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=56b48036033268a00528783adbdab412, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:04,289 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964289"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310964289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310964289"}]},"ts":"1676310964289"} 2023-02-13 17:56:04,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 56b48036033268a00528783adbdab412, server=jenkins-hbase4.apache.org,46577,1676310860699}] 2023-02-13 17:56:04,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 17:56:04,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 56b48036033268a00528783adbdab412, disabling compactions & flushes 2023-02-13 17:56:04,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. after waiting 0 ms 2023-02-13 17:56:04,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,440 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8] to archive 2023-02-13 17:56:04,440 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:04,442 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-95e03b821ad4468392186391879f2cef 2023-02-13 17:56:04,443 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-bc9b1b11d8234502a5c129cb9cada938 2023-02-13 17:56:04,444 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fb49e4110f5a418f8a8822cddf15016a 2023-02-13 17:56:04,445 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b94e913d45a74de5857915fdbd7e2057 2023-02-13 17:56:04,446 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-701506c46c10402890874c5eff3fccb1 2023-02-13 17:56:04,447 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-52bffd3b83574722b30e08ca75fd5da2 2023-02-13 17:56:04,447 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2e825bbd99484113893a212569a8e0d8 2023-02-13 17:56:04,449 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e] to archive 2023-02-13 17:56:04,449 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:04,450 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-e378335f3898494994e57ddb302e371d 2023-02-13 17:56:04,451 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-2b30f3b0965f4cff8acb4440574229fd 2023-02-13 17:56:04,452 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-31f0d99827134638961bfe1c159a6d15 2023-02-13 17:56:04,453 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-fc1505727ff64852a648f07b865ee762 2023-02-13 17:56:04,454 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-b149248029dc44d3b8f90620875ee00f 2023-02-13 17:56:04,455 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-faf2516095d94943868aaee8ae96622c 2023-02-13 17:56:04,455 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/testSplitTableRegionEmptyDaughter=09e0c5ec0772f8303a6c93b7fa1dfc73-0cf33898cfbc44599ab5e1f0e3e5a79e 2023-02-13 17:56:04,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:56:04,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:04,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412. 2023-02-13 17:56:04,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 56b48036033268a00528783adbdab412: Waiting for close lock at 1676310964439Running coprocessor pre-close hooks at 1676310964439Disabling compacts and flushes for region at 1676310964439Disabling writes for close at 1676310964439Writing region close event to WAL at 1676310964456 (+17 ms)Running coprocessor post-close hooks at 1676310964459 (+3 ms)Closed at 1676310964459 2023-02-13 17:56:04,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 56b48036033268a00528783adbdab412 2023-02-13 17:56:04,460 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cae400a17fe2777e2ee20b2518d3e960, disabling compactions & flushes 2023-02-13 17:56:04,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. after waiting 0 ms 2023-02-13 17:56:04,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=56b48036033268a00528783adbdab412, regionState=CLOSED 2023-02-13 17:56:04,461 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964461"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310964461"}]},"ts":"1676310964461"} 2023-02-13 17:56:04,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 17:56:04,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 17:56:04,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 56b48036033268a00528783adbdab412, server=jenkins-hbase4.apache.org,46577,1676310860699 in 172 msec 2023-02-13 17:56:04,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:04,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960. 2023-02-13 17:56:04,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cae400a17fe2777e2ee20b2518d3e960: Waiting for close lock at 1676310964460Running coprocessor pre-close hooks at 1676310964460Disabling compacts and flushes for region at 1676310964460Disabling writes for close at 1676310964461 (+1 ms)Writing region close event to WAL at 1676310964461Running coprocessor post-close hooks at 1676310964464 (+3 ms)Closed at 1676310964464 2023-02-13 17:56:04,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=56b48036033268a00528783adbdab412, UNASSIGN in 179 msec 2023-02-13 17:56:04,465 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=cae400a17fe2777e2ee20b2518d3e960, regionState=CLOSED 2023-02-13 17:56:04,466 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676310964466"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310964466"}]},"ts":"1676310964466"} 2023-02-13 17:56:04,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 17:56:04,468 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure cae400a17fe2777e2ee20b2518d3e960, server=jenkins-hbase4.apache.org,46577,1676310860699 in 180 msec 2023-02-13 17:56:04,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-13 17:56:04,469 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=cae400a17fe2777e2ee20b2518d3e960, UNASSIGN in 183 msec 2023-02-13 17:56:04,469 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310964469"}]},"ts":"1676310964469"} 2023-02-13 17:56:04,470 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 17:56:04,471 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 17:56:04,473 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 193 msec 2023-02-13 17:56:04,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 17:56:04,584 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 17:56:04,584 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,587 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 17:56:04,587 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,591 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:04,591 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412 2023-02-13 17:56:04,591 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,593 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/recovered.edits] 2023-02-13 17:56:04,593 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/recovered.edits] 2023-02-13 17:56:04,593 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/recovered.edits] 2023-02-13 17:56:04,597 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/c2af6732de224d42a6a62e58f66d9fcb to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf1/c2af6732de224d42a6a62e58f66d9fcb 2023-02-13 17:56:04,597 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/2e825bbd99484113893a212569a8e0d8 2023-02-13 17:56:04,599 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/52bffd3b83574722b30e08ca75fd5da2 2023-02-13 17:56:04,600 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/4538a88d70fa4887bad66792b52e56ae to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/cf2/4538a88d70fa4887bad66792b52e56ae 2023-02-13 17:56:04,600 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/recovered.edits/88.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960/recovered.edits/88.seqid 2023-02-13 17:56:04,600 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/701506c46c10402890874c5eff3fccb1 2023-02-13 17:56:04,600 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/cae400a17fe2777e2ee20b2518d3e960 2023-02-13 17:56:04,601 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/95e03b821ad4468392186391879f2cef 2023-02-13 17:56:04,603 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412/recovered.edits/90.seqid 2023-02-13 17:56:04,603 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/b94e913d45a74de5857915fdbd7e2057 2023-02-13 17:56:04,603 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/56b48036033268a00528783adbdab412 2023-02-13 17:56:04,604 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/bc9b1b11d8234502a5c129cb9cada938 2023-02-13 17:56:04,605 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf1/fb49e4110f5a418f8a8822cddf15016a 2023-02-13 17:56:04,607 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/0cf33898cfbc44599ab5e1f0e3e5a79e 2023-02-13 17:56:04,608 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/2b30f3b0965f4cff8acb4440574229fd 2023-02-13 17:56:04,609 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/31f0d99827134638961bfe1c159a6d15 2023-02-13 17:56:04,611 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/b149248029dc44d3b8f90620875ee00f 2023-02-13 17:56:04,612 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/e378335f3898494994e57ddb302e371d 2023-02-13 17:56:04,613 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/faf2516095d94943868aaee8ae96622c 2023-02-13 17:56:04,614 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/cf2/fc1505727ff64852a648f07b865ee762 2023-02-13 17:56:04,616 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73/recovered.edits/85.seqid 2023-02-13 17:56:04,616 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegionEmptyDaughter/09e0c5ec0772f8303a6c93b7fa1dfc73 2023-02-13 17:56:04,617 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 17:56:04,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,620 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 17:56:04,622 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 17:56:04,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,623 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 17:56:04,623 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310964623"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:04,623 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310964623"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:04,623 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310964623"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:04,625 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:56:04,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 09e0c5ec0772f8303a6c93b7fa1dfc73, NAME => 'testSplitTableRegionEmptyDaughter,,1676310961905.09e0c5ec0772f8303a6c93b7fa1dfc73.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 56b48036033268a00528783adbdab412, NAME => 'testSplitTableRegionEmptyDaughter,,1676310963413.56b48036033268a00528783adbdab412.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => cae400a17fe2777e2ee20b2518d3e960, NAME => 'testSplitTableRegionEmptyDaughter,71,1676310963413.cae400a17fe2777e2ee20b2518d3e960.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 17:56:04,625 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 17:56:04,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310964625"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:04,626 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 17:56:04,628 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 17:56:04,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 44 msec 2023-02-13 17:56:04,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 17:56:04,689 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 17:56:04,702 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 376), OpenFileDescriptor=842 (was 844), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=127 (was 127), ProcessCount=168 (was 168), AvailableMemoryMB=832 (was 858) 2023-02-13 17:56:04,715 INFO [Listener at localhost/44157] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=842, MaxFileDescriptor=60000, SystemLoadAverage=127, ProcessCount=168, AvailableMemoryMB=831 2023-02-13 17:56:04,715 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:56:04,718 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 17:56:04,718 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 17:56:04,719 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 17:56:04,720 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 17:56:04,721 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 17:56:04,722 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:04,722 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd empty. 2023-02-13 17:56:04,723 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:04,723 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 17:56:04,735 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 17:56:04,736 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 76ad654509de13740c5dbc13711f86fd, NAME => 'testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 76ad654509de13740c5dbc13711f86fd, disabling compactions & flushes 2023-02-13 17:56:04,746 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. after waiting 0 ms 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:04,746 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:04,746 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 76ad654509de13740c5dbc13711f86fd: Waiting for close lock at 1676310964746Disabling compacts and flushes for region at 1676310964746Disabling writes for close at 1676310964746Writing region close event to WAL at 1676310964746Closed at 1676310964746 2023-02-13 17:56:04,748 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 17:56:04,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310964749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310964749"}]},"ts":"1676310964749"} 2023-02-13 17:56:04,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 17:56:04,750 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 17:56:04,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310964750"}]},"ts":"1676310964750"} 2023-02-13 17:56:04,751 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 17:56:04,754 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 17:56:04,754 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 17:56:04,754 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 17:56:04,754 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 17:56:04,754 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 17:56:04,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, ASSIGN}] 2023-02-13 17:56:04,756 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, ASSIGN 2023-02-13 17:56:04,756 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:56:04,906 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 17:56:04,907 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=76ad654509de13740c5dbc13711f86fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:04,908 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310964907"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310964907"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310964907"}]},"ts":"1676310964907"} 2023-02-13 17:56:04,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 76ad654509de13740c5dbc13711f86fd, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:05,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 76ad654509de13740c5dbc13711f86fd, NAME => 'testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 17:56:05,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:05,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:05,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,066 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,067 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,068 DEBUG [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1 2023-02-13 17:56:05,068 DEBUG [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1 2023-02-13 17:56:05,068 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 76ad654509de13740c5dbc13711f86fd columnFamilyName cf1 2023-02-13 17:56:05,069 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] regionserver.HStore(310): Store=76ad654509de13740c5dbc13711f86fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:05,069 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,070 DEBUG [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2 2023-02-13 17:56:05,070 DEBUG [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2 2023-02-13 17:56:05,071 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 76ad654509de13740c5dbc13711f86fd columnFamilyName cf2 2023-02-13 17:56:05,071 INFO [StoreOpener-76ad654509de13740c5dbc13711f86fd-1] regionserver.HStore(310): Store=76ad654509de13740c5dbc13711f86fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:05,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,072 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:05,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:05,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 17:56:05,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 76ad654509de13740c5dbc13711f86fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67064336, jitterRate=-6.635189056396484E-4}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:05,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 76ad654509de13740c5dbc13711f86fd: Running coprocessor pre-open hook at 1676310965066Writing region info on filesystem at 1676310965066Initializing all the Stores at 1676310965066Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310965067 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310965067Cleaning up temporary data from old regions at 1676310965073 (+6 ms)Cleaning up detritus from prior splits at 1676310965073Running coprocessor post-open hooks at 1676310965077 (+4 ms)Region opened successfully at 1676310965077 2023-02-13 17:56:05,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd., pid=143, masterSystemTime=1676310965060 2023-02-13 17:56:05,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=76ad654509de13740c5dbc13711f86fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310965079"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310965079"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310965079"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310965079"}]},"ts":"1676310965079"} 2023-02-13 17:56:05,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 17:56:05,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 76ad654509de13740c5dbc13711f86fd, server=jenkins-hbase4.apache.org,39639,1676310860746 in 171 msec 2023-02-13 17:56:05,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 17:56:05,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, ASSIGN in 326 msec 2023-02-13 17:56:05,083 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 17:56:05,083 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310965083"}]},"ts":"1676310965083"} 2023-02-13 17:56:05,084 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 17:56:05,086 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 17:56:05,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 367 msec 2023-02-13 17:56:05,225 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:05,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:05,227 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:05,227 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,227 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:05,227 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:05,227 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,227 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,229 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,229 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,229 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,229 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,230 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,230 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,230 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,231 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:05,232 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:05,232 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,232 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:05,233 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,234 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,241 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/d701bf844a58497c958295b7135d4e2a 2023-02-13 17:56:05,255 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/8f18fc814fca4c67bcc6f222f68c779c 2023-02-13 17:56:05,258 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/d701bf844a58497c958295b7135d4e2a as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a 2023-02-13 17:56:05,261 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:56:05,261 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/8f18fc814fca4c67bcc6f222f68c779c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c 2023-02-13 17:56:05,264 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:56:05,265 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 32ms, sequenceid=6, compaction requested=false 2023-02-13 17:56:05,265 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 17:56:05,265 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:05,265 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,265 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,267 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,267 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:05,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bf6e64d[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:05,269 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:05,269 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:05,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,270 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,270 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,272 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,272 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,273 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,276 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,276 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,276 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:05,279 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:05,279 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:05,279 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:05,279 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:05,279 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,280 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,281 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,282 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,283 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,283 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,283 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,286 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,286 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,287 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,387 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:05,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:05,396 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:05,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:05,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:05,397 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:05,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:05,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:05,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,398 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,399 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,399 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,399 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,400 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:05,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,401 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,401 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:05,401 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:05,401 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,402 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:05,403 ERROR [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 14616398-3dff-4b20-9b76-b1842232aebe) service to localhost/127.0.0.1:37271 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,411 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/b87a722040f94236af47a411eeed6a31 2023-02-13 17:56:05,423 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/155cf7877b9f4fb8b153a51061286869 2023-02-13 17:56:05,426 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/b87a722040f94236af47a411eeed6a31 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31 2023-02-13 17:56:05,429 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:56:05,430 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/155cf7877b9f4fb8b153a51061286869 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869 2023-02-13 17:56:05,433 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 17:56:05,433 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 31ms, sequenceid=19, compaction requested=false 2023-02-13 17:56:05,433 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:05,433 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:05,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:05,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,434 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,435 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,435 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:05,437 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a7115c8[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:05,437 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:05,437 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,438 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,438 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,439 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:05,439 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,440 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,441 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,441 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,442 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:05,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:05,444 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:05,444 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:05,444 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:05,445 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:05,445 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,446 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,446 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,447 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,447 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,449 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,449 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,449 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,451 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,452 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,452 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:05,452 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,452 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:05,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,453 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,453 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,453 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,453 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,553 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:05,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:05,563 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:05,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:05,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:05,565 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:05,565 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,565 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:05,565 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:05,565 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,565 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,567 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,567 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,567 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,567 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,568 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,568 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,569 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,570 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,570 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:05,570 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:05,570 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:05,571 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,574 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,574 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,574 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,575 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,580 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/a1b259d5a9044a2f81560a09988a95b3 2023-02-13 17:56:05,589 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/c27ce5fc5b7743e190ef932207d1bbe0 2023-02-13 17:56:05,592 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/a1b259d5a9044a2f81560a09988a95b3 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3 2023-02-13 17:56:05,594 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:56:05,595 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/c27ce5fc5b7743e190ef932207d1bbe0 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0 2023-02-13 17:56:05,597 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 17:56:05,598 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 28ms, sequenceid=32, compaction requested=true 2023-02-13 17:56:05,598 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:05,598 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,598 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:05,598 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:05,598 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,598 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,598 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,601 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,601 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,601 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,601 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:05,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30f91c58[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:05,602 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:05,603 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,603 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,603 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,603 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,604 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:05,605 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,605 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:05,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:05,608 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:05,608 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:05,609 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:05,609 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:05,609 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,611 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,612 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,613 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,614 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,615 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,615 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,618 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,618 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:05,618 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,617 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:05,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:05,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,618 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,619 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,719 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:05,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:05,729 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:05,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:05,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:05,730 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:05,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:05,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:05,730 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,731 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,732 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,732 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,732 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,732 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,733 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,733 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,733 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:05,734 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,734 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,734 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:05,734 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:05,735 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,736 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,736 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,736 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,737 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,737 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,746 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/247231323fbb4f99855b9fe05a8af766 2023-02-13 17:56:05,755 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/0c07bdf5719a4934b315945938090bfc 2023-02-13 17:56:05,758 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/247231323fbb4f99855b9fe05a8af766 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766 2023-02-13 17:56:05,761 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:56:05,762 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/0c07bdf5719a4934b315945938090bfc as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc 2023-02-13 17:56:05,764 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 17:56:05,765 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 31ms, sequenceid=45, compaction requested=true 2023-02-13 17:56:05,765 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:05,765 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,765 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:05,765 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:05,765 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,765 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,765 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,767 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,767 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,767 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,767 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:05,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b65b02c[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:05,769 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:05,769 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:05,770 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,770 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:05,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,770 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,771 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,771 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:05,771 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,772 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,773 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,773 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,773 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,773 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,774 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,774 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,774 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:05,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:05,777 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:05,777 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:05,777 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:05,777 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:05,777 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:05,780 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,780 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,780 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,781 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,781 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,781 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,783 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,784 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,784 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,784 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,784 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,791 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:05,791 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,791 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,790 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,791 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,792 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,792 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,891 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:05,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:05,904 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:05,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:05,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:05,906 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:05,906 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,906 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:05,906 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:05,906 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,906 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,907 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,907 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,907 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,907 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,908 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,908 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,908 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:05,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,909 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,909 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,909 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:05,909 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:05,910 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:05,909 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,910 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,911 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,911 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,911 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,911 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,912 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,912 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,918 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/35a9b15e6eb84ac3821bd9410f381d9d 2023-02-13 17:56:05,934 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/8c0197872c144970a82127e66152dd17 2023-02-13 17:56:05,937 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/35a9b15e6eb84ac3821bd9410f381d9d as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d 2023-02-13 17:56:05,940 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:56:05,941 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/8c0197872c144970a82127e66152dd17 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17 2023-02-13 17:56:05,943 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 17:56:05,944 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 35ms, sequenceid=58, compaction requested=true 2023-02-13 17:56:05,944 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:05,944 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:05,944 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:05,944 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:05,944 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,944 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:05,944 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:05,946 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:05,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67fdfd83[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:05,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:05,948 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,949 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,949 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,949 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,950 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,950 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:05,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:05,952 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,952 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:05,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:05,955 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:05,955 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:05,955 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:05,955 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:05,955 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,958 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:05,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:05,959 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:05,959 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,961 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:05,961 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:05,961 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,964 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:05,964 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:05,964 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:05,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:05,964 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:05,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:05,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:05,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:05,965 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,065 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:06,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:06,078 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 17:56:06,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 17:56:06,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 17:56:06,080 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 17:56:06,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:06,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 17:56:06,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 17:56:06,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,080 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,081 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,081 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,081 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,082 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,082 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,082 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 17:56:06,082 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:06,083 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,083 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 17:56:06,083 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:06,083 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:06,083 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:06,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 17:56:06,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34255,1676310860648' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:06,084 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:06,084 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. started... 2023-02-13 17:56:06,084 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46577,1676310860699' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:06,084 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 17:56:06,086 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,088 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:06,095 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/a882845c67124403b05c3eec32d719fb 2023-02-13 17:56:06,111 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/f0ec7e55864c46088021165b7867b318 2023-02-13 17:56:06,115 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/a882845c67124403b05c3eec32d719fb as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb 2023-02-13 17:56:06,118 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:56:06,119 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/f0ec7e55864c46088021165b7867b318 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318 2023-02-13 17:56:06,122 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 17:56:06,123 INFO [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 39ms, sequenceid=71, compaction requested=true 2023-02-13 17:56:06,123 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 76ad654509de13740c5dbc13711f86fd: 2023-02-13 17:56:06,123 DEBUG [rs(jenkins-hbase4.apache.org,39639,1676310860746)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 17:56:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 17:56:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 17:56:06,123 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 17:56:06,125 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:06,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:06,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,125 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 17:56:06,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:06,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:06,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39639,1676310860746' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 17:56:06,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75307be1[Count = 0] remaining members to acquire global barrier 2023-02-13 17:56:06,127 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 17:56:06,127 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:06,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:06,128 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46577,1676310860699' in zk 2023-02-13 17:56:06,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,128 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 17:56:06,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,39639,1676310860746' in zk 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 17:56:06,129 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34255,1676310860648' in zk 2023-02-13 17:56:06,134 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:06,134 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:06,134 DEBUG [member: 'jenkins-hbase4.apache.org,46577,1676310860699' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:06,135 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 17:56:06,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,39639,1676310860746' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:06,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:06,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:06,135 DEBUG [member: 'jenkins-hbase4.apache.org,34255,1676310860648' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 17:56:06,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:06,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:06,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:06,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,39639,1676310860746': 2023-02-13 17:56:06,138 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39639,1676310860746' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 17:56:06,138 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 17:56:06,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 17:56:06,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 17:56:06,138 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 17:56:06,138 INFO [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 17:56:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,140 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,140 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 17:56:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,141 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,141 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 17:56:06,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 17:56:06,142 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 17:56:06,143 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,143 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,145 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,145 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,145 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,145 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,146 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 17:56:06,146 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 17:56:06,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,146 DEBUG [(jenkins-hbase4.apache.org,33435,1676310859495)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 17:56:06,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 17:56:06,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 17:56:06,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:06,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 17:56:06,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 17:56:06,146 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,147 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,147 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:06,147 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 17:56:06,147 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 17:56:06,246 DEBUG [Listener at localhost/44157] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 17:56:06,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 17:56:06,254 DEBUG [Listener at localhost/44157] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,255 DEBUG [Listener at localhost/44157] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=76ad654509de13740c5dbc13711f86fd, daughterA=3a950ae7e4bc98b49ce2888ab6f93154, daughterB=aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:06,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=76ad654509de13740c5dbc13711f86fd, daughterA=3a950ae7e4bc98b49ce2888ab6f93154, daughterB=aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:06,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=76ad654509de13740c5dbc13711f86fd, daughterA=3a950ae7e4bc98b49ce2888ab6f93154, daughterB=aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:06,256 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=76ad654509de13740c5dbc13711f86fd, daughterA=3a950ae7e4bc98b49ce2888ab6f93154, daughterB=aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:06,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, UNASSIGN}] 2023-02-13 17:56:06,258 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, UNASSIGN 2023-02-13 17:56:06,258 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=76ad654509de13740c5dbc13711f86fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:06,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310966258"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310966258"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310966258"}]},"ts":"1676310966258"} 2023-02-13 17:56:06,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 76ad654509de13740c5dbc13711f86fd, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:06,399 ERROR [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 51672cbd-b28f-41e9-adef-3ec58fde0c7e) service to localhost/127.0.0.1:37271 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 17:56:06,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 76ad654509de13740c5dbc13711f86fd, disabling compactions & flushes 2023-02-13 17:56:06,411 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. after waiting 0 ms 2023-02-13 17:56:06,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 76ad654509de13740c5dbc13711f86fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 17:56:06,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 2023-02-13 17:56:06,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/4a464e7bf68f405b9b697cb2f8618e93 2023-02-13 17:56:06,436 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 2023-02-13 17:56:06,439 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:56:06,440 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/.tmp/cf2/4a464e7bf68f405b9b697cb2f8618e93 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93 2023-02-13 17:56:06,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 17:56:06,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 32ms, sequenceid=82, compaction requested=true 2023-02-13 17:56:06,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 17:56:06,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:06,450 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. 2023-02-13 17:56:06,450 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 76ad654509de13740c5dbc13711f86fd: Waiting for close lock at 1676310966411Running coprocessor pre-close hooks at 1676310966411Disabling compacts and flushes for region at 1676310966411Disabling writes for close at 1676310966412 (+1 ms)Obtaining lock to block concurrent updates at 1676310966412Preparing flush snapshotting stores in 76ad654509de13740c5dbc13711f86fd at 1676310966412Finished memstore snapshotting testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676310966412Flushing stores of testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. at 1676310966412Flushing 76ad654509de13740c5dbc13711f86fd/cf1: creating writer at 1676310966413 (+1 ms)Flushing 76ad654509de13740c5dbc13711f86fd/cf1: appending metadata at 1676310966415 (+2 ms)Flushing 76ad654509de13740c5dbc13711f86fd/cf1: closing flushed file at 1676310966415Flushing 76ad654509de13740c5dbc13711f86fd/cf2: creating writer at 1676310966427 (+12 ms)Flushing 76ad654509de13740c5dbc13711f86fd/cf2: appending metadata at 1676310966428 (+1 ms)Flushing 76ad654509de13740c5dbc13711f86fd/cf2: closing flushed file at 1676310966428Flushing 76ad654509de13740c5dbc13711f86fd/cf1: reopening flushed file at 1676310966437 (+9 ms)Flushing 76ad654509de13740c5dbc13711f86fd/cf2: reopening flushed file at 1676310966440 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 76ad654509de13740c5dbc13711f86fd in 32ms, sequenceid=82, compaction requested=true at 1676310966444 (+4 ms)Writing region close event to WAL at 1676310966447 (+3 ms)Running coprocessor post-close hooks at 1676310966450 (+3 ms)Closed at 1676310966450 2023-02-13 17:56:06,452 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,452 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=76ad654509de13740c5dbc13711f86fd, regionState=CLOSED 2023-02-13 17:56:06,452 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310966452"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310966452"}]},"ts":"1676310966452"} 2023-02-13 17:56:06,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 17:56:06,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 76ad654509de13740c5dbc13711f86fd, server=jenkins-hbase4.apache.org,39639,1676310860746 in 193 msec 2023-02-13 17:56:06,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 17:56:06,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=76ad654509de13740c5dbc13711f86fd, UNASSIGN in 198 msec 2023-02-13 17:56:06,465 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=76ad654509de13740c5dbc13711f86fd, threads=14 2023-02-13 17:56:06,465 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,465 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,465 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,466 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,466 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,466 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,466 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,466 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,467 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,616 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,635 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,641 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,664 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,668 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,680 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,690 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,690 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,692 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,694 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:06,991 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:07,043 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:07,043 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31 for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:07,043 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb for region: 76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:07,043 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 76ad654509de13740c5dbc13711f86fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 17:56:07,084 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:56:07,086 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 17:56:07,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676310967087"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676310967087"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676310967087"}]},"ts":"1676310967087"} 2023-02-13 17:56:07,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310967087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310967087"}]},"ts":"1676310967087"} 2023-02-13 17:56:07,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967087"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310967087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310967087"}]},"ts":"1676310967087"} 2023-02-13 17:56:07,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, ASSIGN}] 2023-02-13 17:56:07,090 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, ASSIGN 2023-02-13 17:56:07,091 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:56:07,091 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, ASSIGN 2023-02-13 17:56:07,092 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,39639,1676310860746; forceNewPlan=false, retain=false 2023-02-13 17:56:07,241 INFO [jenkins-hbase4:33435] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 17:56:07,243 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=3a950ae7e4bc98b49ce2888ab6f93154, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,243 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967243"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310967243"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310967243"}]},"ts":"1676310967243"} 2023-02-13 17:56:07,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 3a950ae7e4bc98b49ce2888ab6f93154, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:07,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=aa2057b995a5df986d2b199593d5918b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967246"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310967246"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310967246"}]},"ts":"1676310967246"} 2023-02-13 17:56:07,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure aa2057b995a5df986d2b199593d5918b, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:07,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3a950ae7e4bc98b49ce2888ab6f93154, NAME => 'testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 17:56:07,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,400 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,400 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1 2023-02-13 17:56:07,401 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1 2023-02-13 17:56:07,401 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a950ae7e4bc98b49ce2888ab6f93154 columnFamilyName cf1 2023-02-13 17:56:07,413 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-bottom 2023-02-13 17:56:07,416 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-bottom 2023-02-13 17:56:07,420 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-bottom 2023-02-13 17:56:07,423 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-bottom 2023-02-13 17:56:07,427 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-bottom 2023-02-13 17:56:07,430 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-bottom 2023-02-13 17:56:07,434 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-bottom 2023-02-13 17:56:07,434 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(310): Store=3a950ae7e4bc98b49ce2888ab6f93154/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:07,434 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,435 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2 2023-02-13 17:56:07,435 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2 2023-02-13 17:56:07,435 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3a950ae7e4bc98b49ce2888ab6f93154 columnFamilyName cf2 2023-02-13 17:56:07,447 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-bottom 2023-02-13 17:56:07,450 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-bottom 2023-02-13 17:56:07,454 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-bottom 2023-02-13 17:56:07,457 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-bottom 2023-02-13 17:56:07,461 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-bottom 2023-02-13 17:56:07,465 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-bottom 2023-02-13 17:56:07,468 DEBUG [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-bottom 2023-02-13 17:56:07,469 INFO [StoreOpener-3a950ae7e4bc98b49ce2888ab6f93154-1] regionserver.HStore(310): Store=3a950ae7e4bc98b49ce2888ab6f93154/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:07,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:07,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:07,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3a950ae7e4bc98b49ce2888ab6f93154; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73204025, jitterRate=0.09082497656345367}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:07,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3a950ae7e4bc98b49ce2888ab6f93154: Running coprocessor pre-open hook at 1676310967399Writing region info on filesystem at 1676310967399Initializing all the Stores at 1676310967399Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310967399Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310967400 (+1 ms)Cleaning up temporary data from old regions at 1676310967471 (+71 ms)Cleaning up detritus from prior splits at 1676310967471Running coprocessor post-open hooks at 1676310967473 (+2 ms)Region opened successfully at 1676310967473 2023-02-13 17:56:07,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154., pid=149, masterSystemTime=1676310967396 2023-02-13 17:56:07,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 17:56:07,475 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:07,475 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,475 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): 3a950ae7e4bc98b49ce2888ab6f93154/cf1 is initiating minor compaction (all files) 2023-02-13 17:56:07,475 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): 3a950ae7e4bc98b49ce2888ab6f93154/cf2 is initiating minor compaction (all files) 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3a950ae7e4bc98b49ce2888ab6f93154/cf1 in testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3a950ae7e4bc98b49ce2888ab6f93154/cf2 in testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/.tmp, totalSize=34.9 K 2023-02-13 17:56:07,475 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-bottom] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/.tmp, totalSize=34.9 K 2023-02-13 17:56:07,476 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310965222 2023-02-13 17:56:07,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:07,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aa2057b995a5df986d2b199593d5918b, NAME => 'testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 17:56:07,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=3a950ae7e4bc98b49ce2888ab6f93154, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,476 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310965388 2023-02-13 17:56:07,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310967476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310967476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310967476"}]},"ts":"1676310967476"} 2023-02-13 17:56:07,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 17:56:07,477 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676310965222 2023-02-13 17:56:07,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 17:56:07,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,477 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676310965388 2023-02-13 17:56:07,477 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310965554 2023-02-13 17:56:07,478 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310965720 2023-02-13 17:56:07,478 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676310965554 2023-02-13 17:56:07,478 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310965893 2023-02-13 17:56:07,478 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,478 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676310965720 2023-02-13 17:56:07,479 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310966067 2023-02-13 17:56:07,479 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676310965893 2023-02-13 17:56:07,479 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310966248 2023-02-13 17:56:07,479 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676310966067 2023-02-13 17:56:07,479 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1 2023-02-13 17:56:07,480 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1 2023-02-13 17:56:07,480 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa2057b995a5df986d2b199593d5918b columnFamilyName cf1 2023-02-13 17:56:07,480 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676310966248 2023-02-13 17:56:07,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-13 17:56:07,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 3a950ae7e4bc98b49ce2888ab6f93154, server=jenkins-hbase4.apache.org,39639,1676310860746 in 234 msec 2023-02-13 17:56:07,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, ASSIGN in 390 msec 2023-02-13 17:56:07,492 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3a950ae7e4bc98b49ce2888ab6f93154#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:07,493 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): 3a950ae7e4bc98b49ce2888ab6f93154#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:07,493 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-top 2023-02-13 17:56:07,496 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-top 2023-02-13 17:56:07,501 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-top 2023-02-13 17:56:07,504 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-top 2023-02-13 17:56:07,504 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/.tmp/cf1/ba1c73372fdb493482dd548f29d74229 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/ba1c73372fdb493482dd548f29d74229 2023-02-13 17:56:07,507 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/.tmp/cf2/82f3f164478f4b44ae6d679ac447c053 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/82f3f164478f4b44ae6d679ac447c053 2023-02-13 17:56:07,507 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-top 2023-02-13 17:56:07,509 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3a950ae7e4bc98b49ce2888ab6f93154/cf1 of 3a950ae7e4bc98b49ce2888ab6f93154 into ba1c73372fdb493482dd548f29d74229(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:07,509 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3a950ae7e4bc98b49ce2888ab6f93154: 2023-02-13 17:56:07,509 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154., storeName=3a950ae7e4bc98b49ce2888ab6f93154/cf1, priority=9, startTime=1676310967474; duration=0sec 2023-02-13 17:56:07,509 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,510 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-top 2023-02-13 17:56:07,511 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3a950ae7e4bc98b49ce2888ab6f93154/cf2 of 3a950ae7e4bc98b49ce2888ab6f93154 into 82f3f164478f4b44ae6d679ac447c053(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:07,511 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3a950ae7e4bc98b49ce2888ab6f93154: 2023-02-13 17:56:07,511 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154., storeName=3a950ae7e4bc98b49ce2888ab6f93154/cf2, priority=9, startTime=1676310967475; duration=0sec 2023-02-13 17:56:07,512 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,513 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-top 2023-02-13 17:56:07,513 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(310): Store=aa2057b995a5df986d2b199593d5918b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:07,514 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,514 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2 2023-02-13 17:56:07,514 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2 2023-02-13 17:56:07,515 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa2057b995a5df986d2b199593d5918b columnFamilyName cf2 2023-02-13 17:56:07,524 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-top 2023-02-13 17:56:07,526 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-top 2023-02-13 17:56:07,529 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-top 2023-02-13 17:56:07,532 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-top 2023-02-13 17:56:07,537 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-top 2023-02-13 17:56:07,540 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-top 2023-02-13 17:56:07,544 DEBUG [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(539): loaded hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-top 2023-02-13 17:56:07,544 INFO [StoreOpener-aa2057b995a5df986d2b199593d5918b-1] regionserver.HStore(310): Store=aa2057b995a5df986d2b199593d5918b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 17:56:07,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 17:56:07,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:07,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aa2057b995a5df986d2b199593d5918b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71447219, jitterRate=0.06464652717113495}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 17:56:07,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aa2057b995a5df986d2b199593d5918b: Running coprocessor pre-open hook at 1676310967477Writing region info on filesystem at 1676310967477Initializing all the Stores at 1676310967478 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310967478Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676310967478Cleaning up temporary data from old regions at 1676310967546 (+68 ms)Cleaning up detritus from prior splits at 1676310967547 (+1 ms)Running coprocessor post-open hooks at 1676310967549 (+2 ms)Region opened successfully at 1676310967549 2023-02-13 17:56:07,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b., pid=150, masterSystemTime=1676310967396 2023-02-13 17:56:07,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,551 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:07,551 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,551 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 17:56:07,551 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,551 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,551 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1912): aa2057b995a5df986d2b199593d5918b/cf2 is initiating minor compaction (all files) 2023-02-13 17:56:07,551 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2258): Starting compaction of aa2057b995a5df986d2b199593d5918b/cf2 in testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,551 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1912): aa2057b995a5df986d2b199593d5918b/cf1 is initiating minor compaction (all files) 2023-02-13 17:56:07,551 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aa2057b995a5df986d2b199593d5918b/cf1 in testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,551 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/.tmp, totalSize=34.9 K 2023-02-13 17:56:07,552 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-top] into tmpdir=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/.tmp, totalSize=34.9 K 2023-02-13 17:56:07,552 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310965222 2023-02-13 17:56:07,552 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676310965222 2023-02-13 17:56:07,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:07,553 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310965388 2023-02-13 17:56:07,553 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676310965388 2023-02-13 17:56:07,553 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310965554 2023-02-13 17:56:07,553 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=aa2057b995a5df986d2b199593d5918b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,553 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967553"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676310967553"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676310967553"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676310967553"}]},"ts":"1676310967553"} 2023-02-13 17:56:07,554 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676310965554 2023-02-13 17:56:07,554 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310965720 2023-02-13 17:56:07,554 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676310965720 2023-02-13 17:56:07,554 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting 35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310965893 2023-02-13 17:56:07,554 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676310965893 2023-02-13 17:56:07,555 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310966067 2023-02-13 17:56:07,555 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676310966067 2023-02-13 17:56:07,555 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] compactions.Compactor(207): Compacting f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310966248 2023-02-13 17:56:07,555 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] compactions.Compactor(207): Compacting 4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676310966248 2023-02-13 17:56:07,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-13 17:56:07,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure aa2057b995a5df986d2b199593d5918b, server=jenkins-hbase4.apache.org,39639,1676310860746 in 308 msec 2023-02-13 17:56:07,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-13 17:56:07,559 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, ASSIGN in 468 msec 2023-02-13 17:56:07,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=76ad654509de13740c5dbc13711f86fd, daughterA=3a950ae7e4bc98b49ce2888ab6f93154, daughterB=aa2057b995a5df986d2b199593d5918b in 1.3050 sec 2023-02-13 17:56:07,568 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa2057b995a5df986d2b199593d5918b#cf1#compaction#144 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:07,568 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] throttle.PressureAwareThroughputController(145): aa2057b995a5df986d2b199593d5918b#cf2#compaction#145 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 17:56:07,580 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/.tmp/cf1/3732d2a372e242448a37981a70a3d625 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/3732d2a372e242448a37981a70a3d625 2023-02-13 17:56:07,581 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/.tmp/cf2/6e71303dce684378bcde0588ab2e7de0 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/6e71303dce684378bcde0588ab2e7de0 2023-02-13 17:56:07,585 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in aa2057b995a5df986d2b199593d5918b/cf1 of aa2057b995a5df986d2b199593d5918b into 3732d2a372e242448a37981a70a3d625(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:07,585 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in aa2057b995a5df986d2b199593d5918b/cf2 of aa2057b995a5df986d2b199593d5918b into 6e71303dce684378bcde0588ab2e7de0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 17:56:07,585 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aa2057b995a5df986d2b199593d5918b: 2023-02-13 17:56:07,585 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for aa2057b995a5df986d2b199593d5918b: 2023-02-13 17:56:07,585 INFO [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b., storeName=aa2057b995a5df986d2b199593d5918b/cf1, priority=9, startTime=1676310967550; duration=0sec 2023-02-13 17:56:07,585 INFO [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b., storeName=aa2057b995a5df986d2b199593d5918b/cf2, priority=9, startTime=1676310967551; duration=0sec 2023-02-13 17:56:07,585 DEBUG [RS:2;jenkins-hbase4:39639-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,585 DEBUG [RS:2;jenkins-hbase4:39639-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 17:56:07,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39639] ipc.CallRunner(144): callId: 1120 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:60440 deadline: 1676311027756, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd. is not online on jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,866 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:56:07,872 INFO [Listener at localhost/44157] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 17:56:07,873 DEBUG [Listener at localhost/44157] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 17:56:07,879 WARN [Listener at localhost/44157] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 17:56:07,880 INFO [Listener at localhost/44157] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 17:56:07,880 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-13 17:56:07,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 17:56:07,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 17:56:07,883 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310967883"}]},"ts":"1676310967883"} 2023-02-13 17:56:07,884 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 17:56:07,887 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 17:56:07,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, UNASSIGN}] 2023-02-13 17:56:07,888 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, UNASSIGN 2023-02-13 17:56:07,888 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=aa2057b995a5df986d2b199593d5918b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967888"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310967888"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310967888"}]},"ts":"1676310967888"} 2023-02-13 17:56:07,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure aa2057b995a5df986d2b199593d5918b, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:07,891 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, UNASSIGN 2023-02-13 17:56:07,891 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=3a950ae7e4bc98b49ce2888ab6f93154, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:07,891 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310967891"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676310967891"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676310967891"}]},"ts":"1676310967891"} 2023-02-13 17:56:07,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 3a950ae7e4bc98b49ce2888ab6f93154, server=jenkins-hbase4.apache.org,39639,1676310860746}] 2023-02-13 17:56:07,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 17:56:08,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:08,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3a950ae7e4bc98b49ce2888ab6f93154, disabling compactions & flushes 2023-02-13 17:56:08,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:08,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:08,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. after waiting 0 ms 2023-02-13 17:56:08,041 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:08,042 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-bottom] to archive 2023-02-13 17:56:08,042 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:08,044 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,045 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,045 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,046 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,047 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,048 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,049 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,050 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-bottom, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-bottom] to archive 2023-02-13 17:56:08,050 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:08,051 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,052 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,053 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,054 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,055 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,055 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,056 DEBUG [StoreCloser-testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:56:08,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:08,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154. 2023-02-13 17:56:08,060 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3a950ae7e4bc98b49ce2888ab6f93154: Waiting for close lock at 1676310968041Running coprocessor pre-close hooks at 1676310968041Disabling compacts and flushes for region at 1676310968041Disabling writes for close at 1676310968041Writing region close event to WAL at 1676310968057 (+16 ms)Running coprocessor post-close hooks at 1676310968060 (+3 ms)Closed at 1676310968060 2023-02-13 17:56:08,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:08,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:08,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aa2057b995a5df986d2b199593d5918b, disabling compactions & flushes 2023-02-13 17:56:08,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:08,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:08,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. after waiting 0 ms 2023-02-13 17:56:08,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:08,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=3a950ae7e4bc98b49ce2888ab6f93154, regionState=CLOSED 2023-02-13 17:56:08,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310968061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310968061"}]},"ts":"1676310968061"} 2023-02-13 17:56:08,062 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641-top] to archive 2023-02-13 17:56:08,063 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:08,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 17:56:08,064 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 3a950ae7e4bc98b49ce2888ab6f93154, server=jenkins-hbase4.apache.org,39639,1676310860746 in 170 msec 2023-02-13 17:56:08,064 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/d701bf844a58497c958295b7135d4e2a.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=3a950ae7e4bc98b49ce2888ab6f93154, UNASSIGN in 177 msec 2023-02-13 17:56:08,065 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/b87a722040f94236af47a411eeed6a31.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,066 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a1b259d5a9044a2f81560a09988a95b3.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,067 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/247231323fbb4f99855b9fe05a8af766.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,068 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/35a9b15e6eb84ac3821bd9410f381d9d.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,069 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/a882845c67124403b05c3eec32d719fb.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,070 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,071 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318-top, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd->hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93-top] to archive 2023-02-13 17:56:08,072 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 17:56:08,073 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8f18fc814fca4c67bcc6f222f68c779c.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,074 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/155cf7877b9f4fb8b153a51061286869.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,075 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/c27ce5fc5b7743e190ef932207d1bbe0.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,076 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/0c07bdf5719a4934b315945938090bfc.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,077 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/8c0197872c144970a82127e66152dd17.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,078 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/f0ec7e55864c46088021165b7867b318.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,079 DEBUG [StoreCloser-testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/4a464e7bf68f405b9b697cb2f8618e93.76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 17:56:08,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:08,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b. 2023-02-13 17:56:08,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aa2057b995a5df986d2b199593d5918b: Waiting for close lock at 1676310968061Running coprocessor pre-close hooks at 1676310968061Disabling compacts and flushes for region at 1676310968061Disabling writes for close at 1676310968061Writing region close event to WAL at 1676310968079 (+18 ms)Running coprocessor post-close hooks at 1676310968082 (+3 ms)Closed at 1676310968082 2023-02-13 17:56:08,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:08,083 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=aa2057b995a5df986d2b199593d5918b, regionState=CLOSED 2023-02-13 17:56:08,083 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676310968083"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676310968083"}]},"ts":"1676310968083"} 2023-02-13 17:56:08,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 17:56:08,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure aa2057b995a5df986d2b199593d5918b, server=jenkins-hbase4.apache.org,39639,1676310860746 in 195 msec 2023-02-13 17:56:08,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 17:56:08,087 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=aa2057b995a5df986d2b199593d5918b, UNASSIGN in 199 msec 2023-02-13 17:56:08,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676310968087"}]},"ts":"1676310968087"} 2023-02-13 17:56:08,088 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 17:56:08,089 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 17:56:08,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 210 msec 2023-02-13 17:56:08,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 17:56:08,186 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 17:56:08,186 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-13 17:56:08,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,189 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,189 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 17:56:08,193 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,193 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:08,193 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:08,194 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/recovered.edits] 2023-02-13 17:56:08,194 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/recovered.edits] 2023-02-13 17:56:08,194 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2, FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/recovered.edits] 2023-02-13 17:56:08,197 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/3732d2a372e242448a37981a70a3d625 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf1/3732d2a372e242448a37981a70a3d625 2023-02-13 17:56:08,197 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/ba1c73372fdb493482dd548f29d74229 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf1/ba1c73372fdb493482dd548f29d74229 2023-02-13 17:56:08,198 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/247231323fbb4f99855b9fe05a8af766 2023-02-13 17:56:08,199 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/35a9b15e6eb84ac3821bd9410f381d9d 2023-02-13 17:56:08,199 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/6e71303dce684378bcde0588ab2e7de0 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/cf2/6e71303dce684378bcde0588ab2e7de0 2023-02-13 17:56:08,200 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/82f3f164478f4b44ae6d679ac447c053 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/cf2/82f3f164478f4b44ae6d679ac447c053 2023-02-13 17:56:08,200 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a1b259d5a9044a2f81560a09988a95b3 2023-02-13 17:56:08,202 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/a882845c67124403b05c3eec32d719fb 2023-02-13 17:56:08,202 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b/recovered.edits/90.seqid 2023-02-13 17:56:08,202 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/recovered.edits/90.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154/recovered.edits/90.seqid 2023-02-13 17:56:08,203 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/aa2057b995a5df986d2b199593d5918b 2023-02-13 17:56:08,203 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/3a950ae7e4bc98b49ce2888ab6f93154 2023-02-13 17:56:08,203 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/b87a722040f94236af47a411eeed6a31 2023-02-13 17:56:08,204 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/d701bf844a58497c958295b7135d4e2a 2023-02-13 17:56:08,205 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf1/f7dc3260e5c44bf9879f1a6d4b5a5641 2023-02-13 17:56:08,207 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/0c07bdf5719a4934b315945938090bfc 2023-02-13 17:56:08,208 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/155cf7877b9f4fb8b153a51061286869 2023-02-13 17:56:08,209 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/4a464e7bf68f405b9b697cb2f8618e93 2023-02-13 17:56:08,210 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8c0197872c144970a82127e66152dd17 2023-02-13 17:56:08,211 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/8f18fc814fca4c67bcc6f222f68c779c 2023-02-13 17:56:08,212 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/c27ce5fc5b7743e190ef932207d1bbe0 2023-02-13 17:56:08,213 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318 to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/cf2/f0ec7e55864c46088021165b7867b318 2023-02-13 17:56:08,214 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/recovered.edits/85.seqid to hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/archive/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd/recovered.edits/85.seqid 2023-02-13 17:56:08,215 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/.tmp/data/default/testSplitTableRegion/76ad654509de13740c5dbc13711f86fd 2023-02-13 17:56:08,215 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 17:56:08,217 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,218 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 17:56:08,219 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 17:56:08,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 17:56:08,220 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310968220"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:08,220 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310968220"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:08,220 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676310968220"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:08,221 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 17:56:08,221 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 76ad654509de13740c5dbc13711f86fd, NAME => 'testSplitTableRegion,,1676310964718.76ad654509de13740c5dbc13711f86fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3a950ae7e4bc98b49ce2888ab6f93154, NAME => 'testSplitTableRegion,,1676310966254.3a950ae7e4bc98b49ce2888ab6f93154.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => aa2057b995a5df986d2b199593d5918b, NAME => 'testSplitTableRegion,41,1676310966254.aa2057b995a5df986d2b199593d5918b.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 17:56:08,222 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 17:56:08,222 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676310968222"}]},"ts":"9223372036854775807"} 2023-02-13 17:56:08,223 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 17:56:08,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 17:56:08,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 37 msec 2023-02-13 17:56:08,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33435] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 17:56:08,290 INFO [Listener at localhost/44157] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 17:56:08,304 INFO [Listener at localhost/44157] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377 (was 375) - Thread LEAK? -, OpenFileDescriptor=844 (was 842) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=133 (was 127) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=819 (was 831) 2023-02-13 17:56:08,304 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 17:56:08,304 INFO [Listener at localhost/44157] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 17:56:08,308 DEBUG [Listener at localhost/44157] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0c9a727f to 127.0.0.1:64222 2023-02-13 17:56:08,308 DEBUG [Listener at localhost/44157] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,310 DEBUG [Listener at localhost/44157] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 17:56:08,310 DEBUG [Listener at localhost/44157] util.JVMClusterUtil(257): Found active master hash=527141208, stopped=false 2023-02-13 17:56:08,310 INFO [Listener at localhost/44157] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:56:08,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 17:56:08,311 INFO [Listener at localhost/44157] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 17:56:08,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:56:08,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 17:56:08,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 17:56:08,311 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 17:56:08,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:56:08,312 DEBUG [Listener at localhost/44157] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x40167ad8 to 127.0.0.1:64222 2023-02-13 17:56:08,312 DEBUG [Listener at localhost/44157] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:56:08,312 INFO [Listener at localhost/44157] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34255,1676310860648' ***** 2023-02-13 17:56:08,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:56:08,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 17:56:08,312 INFO [Listener at localhost/44157] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 17:56:08,313 INFO [Listener at localhost/44157] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,46577,1676310860699' ***** 2023-02-13 17:56:08,313 INFO [Listener at localhost/44157] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 17:56:08,313 INFO [Listener at localhost/44157] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39639,1676310860746' ***** 2023-02-13 17:56:08,313 INFO [Listener at localhost/44157] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 17:56:08,313 INFO [RS:0;jenkins-hbase4:34255] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 17:56:08,313 INFO [RS:2;jenkins-hbase4:39639] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 17:56:08,313 INFO [RS:1;jenkins-hbase4:46577] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 17:56:08,313 INFO [RS:0;jenkins-hbase4:34255] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 17:56:08,313 INFO [RS:2;jenkins-hbase4:39639] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 17:56:08,313 INFO [RS:0;jenkins-hbase4:34255] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 17:56:08,313 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 17:56:08,314 INFO [RS:1;jenkins-hbase4:46577] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 17:56:08,313 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 17:56:08,314 INFO [RS:1;jenkins-hbase4:46577] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 17:56:08,314 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:08,314 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 17:56:08,314 DEBUG [RS:0;jenkins-hbase4:34255] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x52db4343 to 127.0.0.1:64222 2023-02-13 17:56:08,313 INFO [RS:2;jenkins-hbase4:39639] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 17:56:08,314 DEBUG [RS:0;jenkins-hbase4:34255] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,314 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(3304): Received CLOSE for 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:56:08,314 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34255,1676310860648; all regions closed. 2023-02-13 17:56:08,314 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:08,314 DEBUG [RS:2;jenkins-hbase4:39639] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38179274 to 127.0.0.1:64222 2023-02-13 17:56:08,314 DEBUG [RS:2;jenkins-hbase4:39639] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,314 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39639,1676310860746; all regions closed. 2023-02-13 17:56:08,314 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:08,314 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4c0f6985b87a7d952cfd36bf98a5ce33, disabling compactions & flushes 2023-02-13 17:56:08,315 DEBUG [RS:1;jenkins-hbase4:46577] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x55f35cac to 127.0.0.1:64222 2023-02-13 17:56:08,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:56:08,315 DEBUG [RS:1;jenkins-hbase4:46577] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:56:08,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. after waiting 0 ms 2023-02-13 17:56:08,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:56:08,315 INFO [RS:1;jenkins-hbase4:46577] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 17:56:08,315 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 4c0f6985b87a7d952cfd36bf98a5ce33 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 17:56:08,315 INFO [RS:1;jenkins-hbase4:46577] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 17:56:08,315 INFO [RS:1;jenkins-hbase4:46577] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 17:56:08,315 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 17:56:08,315 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-13 17:56:08,315 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1479): Online Regions={4c0f6985b87a7d952cfd36bf98a5ce33=hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33., 1588230740=hbase:meta,,1.1588230740} 2023-02-13 17:56:08,316 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740, 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:56:08,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 17:56:08,317 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 17:56:08,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 17:56:08,317 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 17:56:08,318 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 17:56:08,318 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-13 17:56:08,325 DEBUG [RS:0;jenkins-hbase4:34255] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs 2023-02-13 17:56:08,325 DEBUG [RS:2;jenkins-hbase4:39639] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs 2023-02-13 17:56:08,325 INFO [RS:0;jenkins-hbase4:34255] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34255%2C1676310860648:(num 1676310862271) 2023-02-13 17:56:08,325 DEBUG [RS:0;jenkins-hbase4:34255] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,325 INFO [RS:2;jenkins-hbase4:39639] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39639%2C1676310860746:(num 1676310862271) 2023-02-13 17:56:08,326 DEBUG [RS:2;jenkins-hbase4:39639] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:08,326 INFO [RS:2;jenkins-hbase4:39639] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:08,326 INFO [RS:0;jenkins-hbase4:34255] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:08,326 INFO [RS:0;jenkins-hbase4:34255] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 17:56:08,326 INFO [RS:2;jenkins-hbase4:39639] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 17:56:08,326 INFO [RS:0;jenkins-hbase4:34255] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 17:56:08,326 INFO [RS:0;jenkins-hbase4:34255] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 17:56:08,326 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 17:56:08,327 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 17:56:08,327 INFO [RS:2;jenkins-hbase4:39639] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 17:56:08,327 INFO [RS:2;jenkins-hbase4:39639] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 17:56:08,327 INFO [RS:2;jenkins-hbase4:39639] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 17:56:08,327 INFO [RS:0;jenkins-hbase4:34255] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 17:56:08,329 INFO [RS:0;jenkins-hbase4:34255] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34255 2023-02-13 17:56:08,329 INFO [RS:2;jenkins-hbase4:39639] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39639 2023-02-13 17:56:08,504 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:08,504 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:08,504 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:08,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/.tmp/info/891418b406af4b46835416bc42a8e5ac 2023-02-13 17:56:08,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:56:08,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:08,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:08,513 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39639,1676310860746 2023-02-13 17:56:08,512 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:56:08,513 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:08,513 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:56:08,513 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:08,513 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@21c7d6a5 rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,513 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@24d4591 rejected from java.util.concurrent.ThreadPoolExecutor@672c100[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,513 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34255,1676310860648] 2023-02-13 17:56:08,513 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:56:08,514 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a70b49e rejected from java.util.concurrent.ThreadPoolExecutor@672c100[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,514 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34255,1676310860648 2023-02-13 17:56:08,514 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6293ae02 rejected from java.util.concurrent.ThreadPoolExecutor@672c100[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,514 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34255,1676310860648; numProcessing=1 2023-02-13 17:56:08,516 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34255,1676310860648 already deleted, retry=false 2023-02-13 17:56:08,516 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34255,1676310860648 expired; onlineServers=2 2023-02-13 17:56:08,516 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39639,1676310860746] 2023-02-13 17:56:08,516 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39639,1676310860746; numProcessing=2 2023-02-13 17:56:08,516 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740, 4c0f6985b87a7d952cfd36bf98a5ce33 2023-02-13 17:56:08,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/.tmp/info/891418b406af4b46835416bc42a8e5ac as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/info/891418b406af4b46835416bc42a8e5ac 2023-02-13 17:56:08,517 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39639,1676310860746 already deleted, retry=false 2023-02-13 17:56:08,517 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39639,1676310860746 expired; onlineServers=1 2023-02-13 17:56:08,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/info/891418b406af4b46835416bc42a8e5ac, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 17:56:08,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 4c0f6985b87a7d952cfd36bf98a5ce33 in 207ms, sequenceid=6, compaction requested=false 2023-02-13 17:56:08,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/namespace/4c0f6985b87a7d952cfd36bf98a5ce33/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 17:56:08,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:08,528 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:56:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4c0f6985b87a7d952cfd36bf98a5ce33: Waiting for close lock at 1676310968314Running coprocessor pre-close hooks at 1676310968314Disabling compacts and flushes for region at 1676310968314Disabling writes for close at 1676310968315 (+1 ms)Obtaining lock to block concurrent updates at 1676310968315Preparing flush snapshotting stores in 4c0f6985b87a7d952cfd36bf98a5ce33 at 1676310968315Finished memstore snapshotting hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676310968315Flushing stores of hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. at 1676310968316 (+1 ms)Flushing 4c0f6985b87a7d952cfd36bf98a5ce33/info: creating writer at 1676310968316Flushing 4c0f6985b87a7d952cfd36bf98a5ce33/info: appending metadata at 1676310968322 (+6 ms)Flushing 4c0f6985b87a7d952cfd36bf98a5ce33/info: closing flushed file at 1676310968322Flushing 4c0f6985b87a7d952cfd36bf98a5ce33/info: reopening flushed file at 1676310968518 (+196 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 4c0f6985b87a7d952cfd36bf98a5ce33 in 207ms, sequenceid=6, compaction requested=false at 1676310968522 (+4 ms)Writing region close event to WAL at 1676310968523 (+1 ms)Running coprocessor post-close hooks at 1676310968527 (+4 ms)Closed at 1676310968528 (+1 ms) 2023-02-13 17:56:08,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676310862744.4c0f6985b87a7d952cfd36bf98a5ce33. 2023-02-13 17:56:08,616 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:08,616 INFO [RS:0;jenkins-hbase4:34255] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34255,1676310860648; zookeeper connection closed. 2023-02-13 17:56:08,616 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3c943454 rejected from java.util.concurrent.ThreadPoolExecutor@672c100[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,616 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:34255-0x1004ed1b5780001, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:08,616 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@40a87317] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@40a87317 2023-02-13 17:56:08,616 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7997204c rejected from java.util.concurrent.ThreadPoolExecutor@672c100[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,716 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:08,716 INFO [RS:2;jenkins-hbase4:39639] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39639,1676310860746; zookeeper connection closed. 2023-02-13 17:56:08,716 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4ebd2c31 rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,716 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:39639-0x1004ed1b5780003, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:08,716 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2facabed] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2facabed 2023-02-13 17:56:08,716 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@36fd17bc rejected from java.util.concurrent.ThreadPoolExecutor@25f84645[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:08,717 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 17:56:08,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/info/da7b8d57a8ba48a2a2cedb986b666837 2023-02-13 17:56:08,917 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 17:56:08,918 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da7b8d57a8ba48a2a2cedb986b666837 2023-02-13 17:56:09,107 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-13 17:56:09,107 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-13 17:56:09,107 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 17:56:09,117 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 17:56:09,317 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 17:56:09,317 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 17:56:09,317 DEBUG [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 17:56:09,327 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/rep_barrier/68dcf9b1acff49e2a7a20c1c9ad64f8f 2023-02-13 17:56:09,330 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68dcf9b1acff49e2a7a20c1c9ad64f8f 2023-02-13 17:56:09,339 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/table/543c2016dcf54fc9b004af383a08ba4a 2023-02-13 17:56:09,342 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 543c2016dcf54fc9b004af383a08ba4a 2023-02-13 17:56:09,342 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/info/da7b8d57a8ba48a2a2cedb986b666837 as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/info/da7b8d57a8ba48a2a2cedb986b666837 2023-02-13 17:56:09,345 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for da7b8d57a8ba48a2a2cedb986b666837 2023-02-13 17:56:09,345 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/info/da7b8d57a8ba48a2a2cedb986b666837, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 17:56:09,346 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/rep_barrier/68dcf9b1acff49e2a7a20c1c9ad64f8f as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/rep_barrier/68dcf9b1acff49e2a7a20c1c9ad64f8f 2023-02-13 17:56:09,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 68dcf9b1acff49e2a7a20c1c9ad64f8f 2023-02-13 17:56:09,349 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/rep_barrier/68dcf9b1acff49e2a7a20c1c9ad64f8f, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 17:56:09,349 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/.tmp/table/543c2016dcf54fc9b004af383a08ba4a as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/table/543c2016dcf54fc9b004af383a08ba4a 2023-02-13 17:56:09,352 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 543c2016dcf54fc9b004af383a08ba4a 2023-02-13 17:56:09,352 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/table/543c2016dcf54fc9b004af383a08ba4a, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 17:56:09,352 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 1034ms, sequenceid=210, compaction requested=false 2023-02-13 17:56:09,356 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 17:56:09,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:09,357 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 17:56:09,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 17:56:09,358 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676310968317Running coprocessor pre-close hooks at 1676310968317Disabling compacts and flushes for region at 1676310968317Disabling writes for close at 1676310968318 (+1 ms)Obtaining lock to block concurrent updates at 1676310968319 (+1 ms)Preparing flush snapshotting stores in 1588230740 at 1676310968319Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676310968320 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676310968322 (+2 ms)Flushing 1588230740/info: creating writer at 1676310968322Flushing 1588230740/info: appending metadata at 1676310968330 (+8 ms)Flushing 1588230740/info: closing flushed file at 1676310968330Flushing 1588230740/rep_barrier: creating writer at 1676310968919 (+589 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676310968921 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676310968921Flushing 1588230740/table: creating writer at 1676310969330 (+409 ms)Flushing 1588230740/table: appending metadata at 1676310969333 (+3 ms)Flushing 1588230740/table: closing flushed file at 1676310969333Flushing 1588230740/info: reopening flushed file at 1676310969343 (+10 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676310969346 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1676310969349 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 1034ms, sequenceid=210, compaction requested=false at 1676310969352 (+3 ms)Writing region close event to WAL at 1676310969354 (+2 ms)Running coprocessor post-close hooks at 1676310969357 (+3 ms)Closed at 1676310969357 2023-02-13 17:56:09,358 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 17:56:09,517 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,46577,1676310860699; all regions closed. 2023-02-13 17:56:09,523 DEBUG [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs 2023-02-13 17:56:09,523 INFO [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46577%2C1676310860699.meta:.meta(num 1676310862428) 2023-02-13 17:56:09,529 DEBUG [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/oldWALs 2023-02-13 17:56:09,529 INFO [RS:1;jenkins-hbase4:46577] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46577%2C1676310860699:(num 1676310862271) 2023-02-13 17:56:09,529 DEBUG [RS:1;jenkins-hbase4:46577] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:09,529 INFO [RS:1;jenkins-hbase4:46577] regionserver.LeaseManager(133): Closed leases 2023-02-13 17:56:09,529 INFO [RS:1;jenkins-hbase4:46577] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 17:56:09,530 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 17:56:09,532 INFO [RS:1;jenkins-hbase4:46577] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46577 2023-02-13 17:56:09,535 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46577,1676310860699 2023-02-13 17:56:09,535 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 17:56:09,535 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2ec75a9e rejected from java.util.concurrent.ThreadPoolExecutor@2af8db9e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:09,536 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46577,1676310860699] 2023-02-13 17:56:09,536 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46577,1676310860699; numProcessing=3 2023-02-13 17:56:09,538 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46577,1676310860699 already deleted, retry=false 2023-02-13 17:56:09,538 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46577,1676310860699 expired; onlineServers=0 2023-02-13 17:56:09,538 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33435,1676310859495' ***** 2023-02-13 17:56:09,538 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 17:56:09,539 DEBUG [M:0;jenkins-hbase4:33435] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5e568363, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 17:56:09,539 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33435,1676310859495 2023-02-13 17:56:09,539 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33435,1676310859495; all regions closed. 2023-02-13 17:56:09,539 DEBUG [M:0;jenkins-hbase4:33435] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 17:56:09,539 DEBUG [M:0;jenkins-hbase4:33435] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 17:56:09,539 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 17:56:09,539 DEBUG [M:0;jenkins-hbase4:33435] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 17:56:09,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676310861886] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676310861886,5,FailOnTimeoutGroup] 2023-02-13 17:56:09,539 INFO [M:0;jenkins-hbase4:33435] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 17:56:09,539 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676310861886] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676310861886,5,FailOnTimeoutGroup] 2023-02-13 17:56:09,539 INFO [M:0;jenkins-hbase4:33435] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 17:56:09,540 INFO [M:0;jenkins-hbase4:33435] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 17:56:09,540 DEBUG [M:0;jenkins-hbase4:33435] master.HMaster(1502): Stopping service threads 2023-02-13 17:56:09,540 INFO [M:0;jenkins-hbase4:33435] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 17:56:09,540 ERROR [M:0;jenkins-hbase4:33435] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 17:56:09,540 INFO [M:0;jenkins-hbase4:33435] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 17:56:09,540 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 17:56:09,541 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 17:56:09,541 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 17:56:09,541 DEBUG [M:0;jenkins-hbase4:33435] zookeeper.ZKUtil(398): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 17:56:09,541 WARN [M:0;jenkins-hbase4:33435] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 17:56:09,541 INFO [M:0;jenkins-hbase4:33435] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 17:56:09,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 17:56:09,541 INFO [M:0;jenkins-hbase4:33435] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 17:56:09,542 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 17:56:09,542 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:56:09,542 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:56:09,542 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 17:56:09,542 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:56:09,542 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.94 KB heapSize=718.98 KB 2023-02-13 17:56:09,568 INFO [M:0;jenkins-hbase4:33435] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.94 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7ade59864a5d4865948d2c081f56553c 2023-02-13 17:56:09,571 INFO [M:0;jenkins-hbase4:33435] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ade59864a5d4865948d2c081f56553c 2023-02-13 17:56:09,572 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/7ade59864a5d4865948d2c081f56553c as hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7ade59864a5d4865948d2c081f56553c 2023-02-13 17:56:09,575 INFO [M:0;jenkins-hbase4:33435] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7ade59864a5d4865948d2c081f56553c 2023-02-13 17:56:09,575 INFO [M:0;jenkins-hbase4:33435] regionserver.HStore(1080): Added hdfs://localhost:37271/user/jenkins/test-data/8f067400-9363-5b7e-e1b1-2388331d105e/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/7ade59864a5d4865948d2c081f56553c, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 17:56:09,576 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegion(2947): Finished flush of dataSize ~607.94 KB/622534, heapSize ~718.97 KB/736224, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=1282, compaction requested=false 2023-02-13 17:56:09,576 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 17:56:09,576 DEBUG [M:0;jenkins-hbase4:33435] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676310969542Disabling compacts and flushes for region at 1676310969542Disabling writes for close at 1676310969542Obtaining lock to block concurrent updates at 1676310969542Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676310969542Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622534, getHeapSize=736224, getOffHeapSize=0, getCellsCount=1499 at 1676310969542Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676310969544 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676310969544Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676310969548 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676310969548Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676310969572 (+24 ms)Finished flush of dataSize ~607.94 KB/622534, heapSize ~718.97 KB/736224, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=1282, compaction requested=false at 1676310969576 (+4 ms)Writing region close event to WAL at 1676310969576Closed at 1676310969576 2023-02-13 17:56:09,580 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 17:56:09,580 INFO [M:0;jenkins-hbase4:33435] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 17:56:09,581 INFO [M:0;jenkins-hbase4:33435] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33435 2023-02-13 17:56:09,584 DEBUG [M:0;jenkins-hbase4:33435] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33435,1676310859495 already deleted, retry=false 2023-02-13 17:56:09,637 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:09,637 INFO [RS:1;jenkins-hbase4:46577] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,46577,1676310860699; zookeeper connection closed. 2023-02-13 17:56:09,637 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4ad81a74 rejected from java.util.concurrent.ThreadPoolExecutor@2af8db9e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:09,637 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): regionserver:46577-0x1004ed1b5780002, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:09,637 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@66e83058] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@66e83058 2023-02-13 17:56:09,637 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3217ae75 rejected from java.util.concurrent.ThreadPoolExecutor@2af8db9e[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:09,638 INFO [Listener at localhost/44157] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 17:56:09,648 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 17:56:09,649 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 17:56:09,737 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:09,737 INFO [M:0;jenkins-hbase4:33435] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33435,1676310859495; zookeeper connection closed. 2023-02-13 17:56:09,737 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d79bf74 rejected from java.util.concurrent.ThreadPoolExecutor@1fb6492a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:09,738 DEBUG [Listener at localhost/44157-EventThread] zookeeper.ZKWatcher(600): master:33435-0x1004ed1b5780000, quorum=127.0.0.1:64222, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 17:56:09,738 ERROR [Listener at localhost/44157-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@29620b83 rejected from java.util.concurrent.ThreadPoolExecutor@1fb6492a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 17:56:09,739 WARN [Listener at localhost/44157] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 17:56:09,742 INFO [Listener at localhost/44157] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 17:56:09,847 WARN [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 51672cbd-b28f-41e9-adef-3ec58fde0c7e) service to localhost/127.0.0.1:37271 2023-02-13 17:56:09,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data5/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:09,850 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data6/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:09,851 WARN [Listener at localhost/44157] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 17:56:09,853 INFO [Listener at localhost/44157] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 17:56:09,955 WARN [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 14616398-3dff-4b20-9b76-b1842232aebe) service to localhost/127.0.0.1:37271 2023-02-13 17:56:09,956 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data3/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:09,957 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data4/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:09,958 WARN [Listener at localhost/44157] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 17:56:09,960 INFO [Listener at localhost/44157] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 17:56:10,069 WARN [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 17:56:10,069 WARN [BP-49835195-172.31.14.131-1676310856427 heartbeating to localhost/127.0.0.1:37271] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-49835195-172.31.14.131-1676310856427 (Datanode Uuid 9eb8ad26-3d72-4195-a2ef-90d725f84f89) service to localhost/127.0.0.1:37271 2023-02-13 17:56:10,070 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data1/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:10,070 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7ee41ece-8812-f0d8-d3a3-881b1daf433b/cluster_d14e37b6-5189-c22a-9ce1-050cc27caf1c/dfs/data/data2/current/BP-49835195-172.31.14.131-1676310856427] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 17:56:10,092 INFO [Listener at localhost/44157] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 17:56:10,285 INFO [Listener at localhost/44157] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 17:56:10,556 INFO [Listener at localhost/44157] hbase.HBaseTestingUtility(1293): Minicluster is down